# Neo4j Knowledge Graph Complete Tutorial
## From Zero to Production Graph RAG - 100+ Page Practical Guide

**Version 2.0 | January 2025**  
**For Hackathons, Students, and Production Applications**

---

# Table of Contents

## Part I: Foundations (Sections 1-10)
1. [Introduction to Graph Databases](#1-introduction-to-graph-databases)
2. [Environment Setup](#2-environment-setup)
3. [Nodes, Labels, and Properties](#3-nodes-labels-and-properties)
4. [Relationships and Patterns](#4-relationships-and-patterns)
5. [Basic CRUD Operations](#5-basic-crud-operations)
6. [Advanced Cypher Queries](#6-advanced-cypher-queries)
7. [Indexing and Constraints](#7-indexing-and-constraints)
8. [Transaction Management](#8-transaction-management)
9. [Batch Operations and Import](#9-batch-operations-and-import)
10. [Schema Design Patterns](#10-schema-design-patterns)

## Part II: Graph Data Science (Sections 11-17)
11. [GDS Library Fundamentals](#11-gds-library-fundamentals)
12. [Centrality Algorithms](#12-centrality-algorithms)
13. [Community Detection](#13-community-detection)
14. [Pathfinding Algorithms](#14-pathfinding-algorithms)
15. [Similarity Algorithms](#15-similarity-algorithms)
16. [Link Prediction](#16-link-prediction)
17. [Graph Embeddings](#17-graph-embeddings)

## Part III: Knowledge Graphs (Sections 18-25)
18. [Ontology Design](#18-ontology-design)
19. [Populating Knowledge Graphs](#19-populating-knowledge-graphs)
20. [Document Chunking Strategies](#20-document-chunking-strategies)
21. [Vector Embeddings](#21-vector-embeddings)
22. [HNSW Vector Indexing](#22-hnsw-vector-indexing)
23. [Graph RAG Implementation](#23-graph-rag-implementation)
24. [Advanced Graph RAG Patterns](#24-advanced-graph-rag-patterns)
25. [LLM Integration](#25-llm-integration)

## Part IV: Production Systems (Sections 26-30)
26. [Large-Scale Graph Clustering](#26-large-scale-graph-clustering)
27. [Neo4j Internal Architecture](#27-neo4j-internal-architecture)
28. [Performance Optimization](#28-performance-optimization)
29. [Enterprise Deployment](#29-enterprise-deployment)
30. [REST API Development](#30-rest-api-development)

## Appendices
- [A. Complete Code Reference](#appendix-a-complete-code-reference)
- [B. Cypher Cheat Sheet](#appendix-b-cypher-cheat-sheet)
- [C. Troubleshooting Guide](#appendix-c-troubleshooting-guide)

---

# Part I: Foundations

---

# 1. Introduction to Graph Databases

## 1.1 What is a Graph Database?

A graph database is a database designed to treat relationships between data as equally important as the data itself. It stores data as **nodes** (entities), **relationships** (connections between entities), and **properties** (attributes of nodes and relationships).

### The Property Graph Model

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                         PROPERTY GRAPH MODEL                                │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│                            ┌─────────────────────┐                          │
│                            │       NODE          │                          │
│                            │  Labels: [:Patient] │                          │
│                            │                     │                          │
│                            │  Properties:        │                          │
│                            │    id: "P001"       │                          │
│                            │    name: "John"     │                          │
│                            │    age: 45          │                          │
│                            │    active: true     │                          │
│                            └──────────┬──────────┘                          │
│                                       │                                     │
│                                       │                                     │
│                    ┌──────────────────┴───────────────────┐                 │
│                    │         RELATIONSHIP                 │                 │
│                    │  Type: DIAGNOSED_WITH                │                 │
│                    │  Direction: Patient → Disease        │                 │
│                    │                                      │                 │
│                    │  Properties:                         │                 │
│                    │    diagnosedDate: 2024-01-15         │                 │
│                    │    severity: "moderate"              │                 │
│                    │    diagnosedBy: "Dr. Smith"          │                 │
│                    └──────────────────┬───────────────────┘                 │
│                                       │                                     │
│                                       ▼                                     │
│                            ┌─────────────────────┐                          │
│                            │       NODE          │                          │
│                            │  Labels: [:Disease] │                          │
│                            │                     │                          │
│                            │  Properties:        │                          │
│                            │    id: "E11"        │                          │
│                            │    name: "Diabetes" │                          │
│                            │    category: "Endo" │                          │
│                            └─────────────────────┘                          │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘
```

### Graph vs Relational Database Comparison

| Aspect | Relational (SQL) | Graph (Neo4j) |
|--------|------------------|---------------|
| **Data Model** | Tables with rows and columns | Nodes, relationships, properties |
| **Schema** | Fixed, predefined schema | Flexible, schema-optional |
| **Relationships** | Foreign keys, JOINs at runtime | First-class citizens, pre-computed |
| **JOIN Performance** | Degrades with depth (O(n²) or worse) | Constant for local queries |
| **Query Language** | SQL | Cypher |
| **Best For** | Structured, tabular data | Connected, relationship-rich data |

### When to Use a Graph Database

**Use a graph database when:**

1. **Relationships matter more than individual records**
   - Social networks (friends, followers)
   - Recommendation engines
   - Fraud detection networks

2. **Data is highly interconnected**
   - Knowledge graphs
   - Network topology
   - Supply chain management

3. **Query patterns involve traversal**
   - "Find all friends of friends who like jazz"
   - "What's the shortest path between A and B?"
   - "Find communities in the network"

4. **Schema evolves frequently**
   - Prototyping and agile development
   - Research applications
   - Multi-tenant systems

## 1.2 Why Neo4j?

Neo4j is the world's leading graph database platform, trusted by thousands of organizations including NASA, eBay, and Walmart.

### Key Features

| Feature | Description |
|---------|-------------|
| **Native Graph Storage** | Data stored as graphs from the ground up, not retrofitted |
| **Cypher Query Language** | Intuitive, ASCII-art pattern matching syntax |
| **ACID Transactions** | Full transactional guarantees for data integrity |
| **Graph Data Science (GDS)** | 70+ built-in graph algorithms |
| **Vector Search** | Native HNSW indexes since version 5.11 |
| **Horizontal Scaling** | Cluster mode with causal consistency |

### Neo4j Editions

| Feature | Community | Enterprise |
|---------|-----------|------------|
| Core Database | ✅ | ✅ |
| APOC Procedures | ✅ | ✅ |
| Graph Data Science | Limited | Full |
| Clustering | ❌ | ✅ |
| Role-based Security | Basic | Advanced |
| Hot Backup | ❌ | ✅ |

## 1.3 Core Concepts Deep Dive

### Nodes

Nodes are the fundamental entities in your graph. Think of them as the "nouns" of your data model.

**Key characteristics:**
- Can have zero or more **labels** (like tags or types)
- Can have zero or more **properties** (key-value pairs)
- Are uniquely identified by an internal ID

```cypher
-- Example node
(:Patient {id: "P001", name: "John Doe", age: 45})

-- Node with multiple labels
(:Patient:VIP:Insured {id: "P002", name: "Jane Smith"})
```

### Relationships

Relationships connect nodes and represent the "verbs" of your data model.

**Key characteristics:**
- Always have a **direction** (start node → end node)
- Always have exactly one **type**
- Can have zero or more **properties**
- Can be traversed in either direction in queries

```cypher
-- Example relationship
(patient)-[:DIAGNOSED_WITH {date: date(), severity: "moderate"}]->(disease)
```

### Properties

Properties store the actual data on nodes and relationships.

**Supported property types:**

| Type | Example | Notes |
|------|---------|-------|
| String | `"John Doe"` | UTF-8 encoded |
| Integer | `42` | 64-bit signed |
| Float | `3.14159` | 64-bit IEEE 754 |
| Boolean | `true`/`false` | |
| Date | `date('2024-01-15')` | ISO 8601 |
| Time | `time('12:30:00')` | With timezone |
| LocalTime | `localtime('12:30:00')` | No timezone |
| DateTime | `datetime()` | Full timestamp |
| LocalDateTime | `localdatetime()` | No timezone |
| Duration | `duration('P1M2D')` | ISO 8601 duration |
| Point | `point({x: 1, y: 2})` | 2D or 3D spatial |
| List | `['a', 'b', 'c']` | Homogeneous types |

---

# 2. Environment Setup

## 2.1 Docker Installation (Recommended)

Docker provides the easiest way to run Neo4j with all required plugins.

### Step 1: Create Project Structure

```bash
mkdir -p neo4j-tutorial/src/utils
mkdir -p neo4j-tutorial/data
mkdir -p neo4j-tutorial/docs
cd neo4j-tutorial
```

### Step 2: Create docker-compose.yml

Create the file `docker-compose.yml`:

```yaml
version: '3.8'

services:
  neo4j:
    image: neo4j:5.15.0-enterprise
    container_name: neo4j-tutorial
    ports:
      - "7474:7474"    # HTTP (Browser)
      - "7687:7687"    # Bolt Protocol
    environment:
      # Authentication
      - NEO4J_AUTH=neo4j/password123
      
      # License (required for Enterprise)
      - NEO4J_ACCEPT_LICENSE_AGREEMENT=yes
      
      # Plugins
      - NEO4J_PLUGINS=["graph-data-science", "apoc"]
      
      # Memory Configuration
      - NEO4J_dbms_memory_pagecache_size=2G
      - NEO4J_dbms_memory_heap_initial__size=1G
      - NEO4J_dbms_memory_heap_max__size=2G
      
      # Security for procedures
      - NEO4J_dbms_security_procedures_unrestricted=gds.*,apoc.*
      - NEO4J_dbms_security_procedures_allowlist=gds.*,apoc.*
      
      # Vector index settings (Neo4j 5.11+)
      - NEO4J_dbms_security_procedures_unrestricted=db.index.vector.*
      
    volumes:
      - neo4j_data:/data
      - neo4j_logs:/logs
      - neo4j_plugins:/plugins
    healthcheck:
      test: ["CMD", "wget", "-q", "--spider", "http://localhost:7474"]
      interval: 10s
      timeout: 5s
      retries: 10
      start_period: 30s
    restart: unless-stopped

volumes:
  neo4j_data:
  neo4j_logs:
  neo4j_plugins:
```

### Step 3: Start Neo4j

```bash
# Start in detached mode
docker-compose up -d

# View logs
docker-compose logs -f neo4j

# Wait for "Started." message (usually 30-60 seconds)
```

### Step 4: Verify Installation

Open http://localhost:7474 in your browser.

- **Connect URL:** `neo4j://localhost:7687`
- **Username:** `neo4j`
- **Password:** `password123`

Test with a query:
```cypher
RETURN "Hello, Neo4j!" AS message
```

## 2.2 Python Environment Setup

### Step 1: Create Virtual Environment

```bash
# Create virtual environment
python -m venv venv

# Activate (Linux/Mac)
source venv/bin/activate

# Activate (Windows)
venv\Scripts\activate
```

### Step 2: Create requirements.txt

```text
# Neo4j Driver and GDS
neo4j==5.15.0
graphdatascience==1.7.0

# Vector Embeddings
sentence-transformers==2.2.2
openai==1.6.1
tiktoken==0.5.2

# Data Processing
pandas==2.1.4
numpy==1.26.3

# API Server
fastapi==0.109.0
uvicorn==0.25.0
pydantic==2.5.3

# Utilities
python-dotenv==1.0.0
tqdm==4.66.1

# Testing
pytest==7.4.4
pytest-asyncio==0.23.3
```

### Step 3: Install Dependencies

```bash
pip install -r requirements.txt
```

## 2.3 Environment Configuration

### Create .env File

```bash
# .env
NEO4J_URI=bolt://localhost:7687
NEO4J_USERNAME=neo4j
NEO4J_PASSWORD=password123
NEO4J_DATABASE=neo4j

# Optional: For OpenAI embeddings
OPENAI_API_KEY=sk-your-api-key-here
```

### Create .env.example (for sharing)

```bash
# .env.example
NEO4J_URI=bolt://localhost:7687
NEO4J_USERNAME=neo4j
NEO4J_PASSWORD=your_password_here
NEO4J_DATABASE=neo4j
OPENAI_API_KEY=sk-your-api-key-here
```

## 2.4 Connection Utility Module

Create `src/utils/__init__.py`:

```python
"""Utility modules for Neo4j tutorial."""
```

Create `src/utils/connection.py`:

```python
"""
Neo4j Connection Manager
========================
Centralized connection management with pooling, health checks, and utilities.

Usage:
    from src.utils.connection import get_connection, get_driver
    
    # Option 1: Use singleton connection
    conn = get_connection()
    results = conn.execute_query("MATCH (n) RETURN count(n)")
    
    # Option 2: Get driver for direct session management
    driver = get_driver()
    with driver.session() as session:
        result = session.run("MATCH (n) RETURN count(n)")
        
    # Option 3: Context manager
    with Neo4jConnection() as conn:
        results = conn.execute_query("MATCH (n) RETURN n LIMIT 10")
"""

import os
import logging
from typing import Optional, Dict, Any, List, Union
from contextlib import contextmanager
from dotenv import load_dotenv
from neo4j import GraphDatabase, Driver, Session, Result
from neo4j.exceptions import ServiceUnavailable, AuthError

# Load environment variables
load_dotenv()

# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)


class Neo4jConnection:
    """
    Neo4j connection manager with connection pooling and utilities.
    
    Features:
    - Connection pooling (configurable pool size)
    - Automatic reconnection
    - Health checks
    - Query execution helpers
    - Transaction support
    
    Example:
        >>> conn = Neo4jConnection()
        >>> conn.connect()
        >>> results = conn.execute_query("MATCH (n:Patient) RETURN n LIMIT 5")
        >>> for record in results:
        ...     print(record)
        >>> conn.close()
    """
    
    _instance: Optional['Neo4jConnection'] = None
    
    def __init__(
        self,
        uri: str = None,
        username: str = None,
        password: str = None,
        database: str = None,
        max_connection_pool_size: int = 50,
        connection_timeout: int = 30
    ):
        """
        Initialize connection configuration.
        
        Args:
            uri: Neo4j URI (default: from NEO4J_URI env var)
            username: Username (default: from NEO4J_USERNAME env var)
            password: Password (default: from NEO4J_PASSWORD env var)
            database: Database name (default: from NEO4J_DATABASE env var)
            max_connection_pool_size: Maximum connections in pool
            connection_timeout: Connection timeout in seconds
        """
        self.uri = uri or os.getenv("NEO4J_URI", "bolt://localhost:7687")
        self.username = username or os.getenv("NEO4J_USERNAME", "neo4j")
        self.password = password or os.getenv("NEO4J_PASSWORD", "password123")
        self.database = database or os.getenv("NEO4J_DATABASE", "neo4j")
        self.max_connection_pool_size = max_connection_pool_size
        self.connection_timeout = connection_timeout
        self._driver: Optional[Driver] = None
    
    def connect(self) -> 'Neo4jConnection':
        """
        Establish connection to Neo4j database.
        
        Returns:
            Self for method chaining
            
        Raises:
            AuthError: If authentication fails
            ServiceUnavailable: If Neo4j is not reachable
        """
        if self._driver is None:
            try:
                self._driver = GraphDatabase.driver(
                    self.uri,
                    auth=(self.username, self.password),
                    max_connection_pool_size=self.max_connection_pool_size,
                    connection_timeout=self.connection_timeout
                )
                # Verify connectivity
                self._driver.verify_connectivity()
                logger.info(f"Connected to Neo4j at {self.uri}")
            except AuthError as e:
                logger.error(f"Authentication failed: {e}")
                raise
            except ServiceUnavailable as e:
                logger.error(f"Neo4j not available at {self.uri}: {e}")
                raise
        return self
    
    def close(self) -> None:
        """Close the connection and release resources."""
        if self._driver is not None:
            self._driver.close()
            self._driver = None
            logger.info("Neo4j connection closed")
    
    @property
    def driver(self) -> Driver:
        """
        Get the Neo4j driver, connecting if necessary.
        
        Returns:
            Neo4j Driver instance
        """
        if self._driver is None:
            self.connect()
        return self._driver
    
    def __enter__(self) -> 'Neo4jConnection':
        """Context manager entry."""
        return self.connect()
    
    def __exit__(self, exc_type, exc_val, exc_tb) -> None:
        """Context manager exit."""
        self.close()
    
    @contextmanager
    def session(self, database: str = None) -> Session:
        """
        Get a session with automatic cleanup.
        
        Args:
            database: Optional database override
            
        Yields:
            Neo4j Session
            
        Example:
            with conn.session() as session:
                result = session.run("MATCH (n) RETURN n")
        """
        session = self.driver.session(database=database or self.database)
        try:
            yield session
        finally:
            session.close()
    
    def execute_query(
        self,
        query: str,
        parameters: Dict[str, Any] = None,
        database: str = None
    ) -> List[Dict[str, Any]]:
        """
        Execute a Cypher query and return results as dictionaries.
        
        Args:
            query: Cypher query string
            parameters: Query parameters
            database: Optional database override
            
        Returns:
            List of result records as dictionaries
            
        Example:
            results = conn.execute_query(
                "MATCH (p:Patient) WHERE p.age > $min_age RETURN p",
                {"min_age": 40}
            )
        """
        with self.session(database) as session:
            result = session.run(query, parameters or {})
            return [record.data() for record in result]
    
    def execute_write(
        self,
        query: str,
        parameters: Dict[str, Any] = None,
        database: str = None
    ) -> Dict[str, Any]:
        """
        Execute a write query and return summary.
        
        Args:
            query: Cypher write query
            parameters: Query parameters
            database: Optional database override
            
        Returns:
            Query summary with counters
        """
        with self.session(database) as session:
            result = session.run(query, parameters or {})
            summary = result.consume()
            return {
                "nodes_created": summary.counters.nodes_created,
                "nodes_deleted": summary.counters.nodes_deleted,
                "relationships_created": summary.counters.relationships_created,
                "relationships_deleted": summary.counters.relationships_deleted,
                "properties_set": summary.counters.properties_set,
                "labels_added": summary.counters.labels_added,
                "labels_removed": summary.counters.labels_removed
            }
    
    def health_check(self) -> Dict[str, Any]:
        """
        Perform comprehensive health check.
        
        Returns:
            Health status with version, node count, and connection info
        """
        try:
            self.driver.verify_connectivity()
            
            # Get database info
            with self.session() as session:
                # Get version
                version_result = session.run("CALL dbms.components()")
                version_record = version_result.single()
                version = version_record["versions"][0] if version_record else "unknown"
                
                # Get node count
                count_result = session.run("MATCH (n) RETURN count(n) as count")
                node_count = count_result.single()["count"]
                
                # Get relationship count
                rel_result = session.run("MATCH ()-[r]->() RETURN count(r) as count")
                rel_count = rel_result.single()["count"]
            
            return {
                "status": "healthy",
                "uri": self.uri,
                "database": self.database,
                "version": version,
                "node_count": node_count,
                "relationship_count": rel_count
            }
        except Exception as e:
            return {
                "status": "unhealthy",
                "error": str(e),
                "uri": self.uri
            }
    
    def get_schema(self) -> Dict[str, Any]:
        """
        Get database schema information.
        
        Returns:
            Schema with labels, relationship types, and indexes
        """
        schema = {"labels": [], "relationship_types": [], "indexes": [], "constraints": []}
        
        with self.session() as session:
            # Get labels
            labels_result = session.run("CALL db.labels()")
            schema["labels"] = [r["label"] for r in labels_result]
            
            # Get relationship types
            rel_result = session.run("CALL db.relationshipTypes()")
            schema["relationship_types"] = [r["relationshipType"] for r in rel_result]
            
            # Get indexes
            idx_result = session.run("SHOW INDEXES")
            schema["indexes"] = [
                {"name": r["name"], "type": r["type"], "labels": r["labelsOrTypes"]}
                for r in idx_result
            ]
            
            # Get constraints
            const_result = session.run("SHOW CONSTRAINTS")
            schema["constraints"] = [
                {"name": r["name"], "type": r["type"]}
                for r in const_result
            ]
        
        return schema


# ============================================================
# SINGLETON ACCESS FUNCTIONS
# ============================================================

_connection: Optional[Neo4jConnection] = None


def get_connection() -> Neo4jConnection:
    """
    Get singleton Neo4jConnection instance.
    
    Returns:
        Singleton Neo4jConnection instance
        
    Example:
        conn = get_connection()
        results = conn.execute_query("MATCH (n) RETURN n LIMIT 10")
    """
    global _connection
    if _connection is None:
        _connection = Neo4jConnection()
        _connection.connect()
    return _connection


def get_driver() -> Driver:
    """
    Get Neo4j driver from singleton connection.
    
    Returns:
        Neo4j Driver instance
        
    Example:
        driver = get_driver()
        with driver.session() as session:
            result = session.run("MATCH (n) RETURN count(n)")
    """
    return get_connection().driver


def close_connection() -> None:
    """Close the singleton connection."""
    global _connection
    if _connection is not None:
        _connection.close()
        _connection = None


# ============================================================
# DEMONSTRATION
# ============================================================

if __name__ == "__main__":
    print("=" * 60)
    print("Neo4j Connection Manager - Test")
    print("=" * 60)
    
    try:
        # Test singleton
        conn = get_connection()
        
        # Health check
        print("\n📊 Health Check:")
        health = conn.health_check()
        for key, value in health.items():
            print(f"   {key}: {value}")
        
        # Test query
        print("\n📊 Test Query:")
        result = conn.execute_query("RETURN 'Hello Neo4j!' AS message")
        print(f"   Result: {result}")
        
        # Get schema
        print("\n📊 Schema:")
        schema = conn.get_schema()
        print(f"   Labels: {schema['labels']}")
        print(f"   Relationship Types: {schema['relationship_types']}")
        print(f"   Indexes: {len(schema['indexes'])}")
        
        print("\n✅ Connection test successful!")
        
    except Exception as e:
        print(f"\n❌ Error: {e}")
    finally:
        close_connection()
```

## 2.5 Verify Complete Setup

Create `test_setup.py`:

```python
#!/usr/bin/env python3
"""
Test script to verify Neo4j setup is complete and working.
Run this after docker-compose up and pip install.
"""

import sys
sys.path.insert(0, '.')

from src.utils.connection import get_connection, close_connection


def test_setup():
    """Run all setup verification tests."""
    print("=" * 60)
    print("Neo4j Setup Verification")
    print("=" * 60)
    
    tests_passed = 0
    tests_failed = 0
    
    # Test 1: Connection
    print("\n[1/5] Testing database connection...")
    try:
        conn = get_connection()
        print("   ✅ Connected successfully")
        tests_passed += 1
    except Exception as e:
        print(f"   ❌ Connection failed: {e}")
        tests_failed += 1
        return
    
    # Test 2: Health check
    print("\n[2/5] Running health check...")
    health = conn.health_check()
    if health["status"] == "healthy":
        print(f"   ✅ Database healthy (version: {health['version']})")
        tests_passed += 1
    else:
        print(f"   ❌ Health check failed: {health.get('error')}")
        tests_failed += 1
    
    # Test 3: Write test
    print("\n[3/5] Testing write operations...")
    try:
        conn.execute_query("CREATE (t:TestNode {name: 'test'}) RETURN t")
        conn.execute_query("MATCH (t:TestNode {name: 'test'}) DELETE t")
        print("   ✅ Write operations working")
        tests_passed += 1
    except Exception as e:
        print(f"   ❌ Write failed: {e}")
        tests_failed += 1
    
    # Test 4: GDS availability
    print("\n[4/5] Checking Graph Data Science library...")
    try:
        result = conn.execute_query("RETURN gds.version() AS version")
        if result:
            print(f"   ✅ GDS available (version: {result[0]['version']})")
            tests_passed += 1
        else:
            print("   ⚠️ GDS query returned no results")
            tests_failed += 1
    except Exception as e:
        print(f"   ❌ GDS not available: {e}")
        tests_failed += 1
    
    # Test 5: APOC availability
    print("\n[5/5] Checking APOC library...")
    try:
        result = conn.execute_query("RETURN apoc.version() AS version")
        if result:
            print(f"   ✅ APOC available (version: {result[0]['version']})")
            tests_passed += 1
        else:
            print("   ⚠️ APOC query returned no results")
            tests_failed += 1
    except Exception as e:
        print(f"   ❌ APOC not available: {e}")
        tests_failed += 1
    
    # Summary
    print("\n" + "=" * 60)
    print(f"Results: {tests_passed} passed, {tests_failed} failed")
    print("=" * 60)
    
    if tests_failed == 0:
        print("\n🎉 All tests passed! Your environment is ready.")
    else:
        print("\n⚠️ Some tests failed. Check the errors above.")
    
    close_connection()


if __name__ == "__main__":
    test_setup()
```

Run the test:

```bash
python test_setup.py
```

Expected output:

```
============================================================
Neo4j Setup Verification
============================================================

[1/5] Testing database connection...
   ✅ Connected successfully

[2/5] Running health check...
   ✅ Database healthy (version: 5.15.0)

[3/5] Testing write operations...
   ✅ Write operations working

[4/5] Checking Graph Data Science library...
   ✅ GDS available (version: 2.6.0)

[5/5] Checking APOC library...
   ✅ APOC available (version: 5.15.0)

============================================================
Results: 5 passed, 0 failed
============================================================

🎉 All tests passed! Your environment is ready.
```

---

# 3. Nodes, Labels, and Properties

## 3.1 Understanding Nodes

Nodes are the primary entities in a graph database. In a healthcare context, nodes represent entities like patients, diseases, medications, doctors, and hospitals.

### Node Anatomy Diagram

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                              NODE STRUCTURE                                 │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│   ┌─────────────────────────────────────────────────────────────────────┐  │
│   │                           LABELS                                     │  │
│   │                                                                      │  │
│   │   [:Patient]  [:VIP]  [:Insured]                                    │  │
│   │                                                                      │  │
│   │   • Nodes can have zero or more labels                              │  │
│   │   • Labels categorize nodes (like types or tags)                    │  │
│   │   • Labels are used for indexing and queries                        │  │
│   │   • Convention: PascalCase                                          │  │
│   └─────────────────────────────────────────────────────────────────────┘  │
│                                                                             │
│   ┌─────────────────────────────────────────────────────────────────────┐  │
│   │                          PROPERTIES                                  │  │
│   │                                                                      │  │
│   │   id: "P001"              (String - unique identifier)              │  │
│   │   name: "John Doe"        (String)                                  │  │
│   │   dateOfBirth: date()     (Date)                                    │  │
│   │   age: 45                 (Integer)                                 │  │
│   │   weight: 82.5            (Float)                                   │  │
│   │   active: true            (Boolean)                                 │  │
│   │   allergies: [...]        (List)                                    │  │
│   │   location: point()       (Spatial)                                 │  │
│   │   createdAt: datetime()   (DateTime)                                │  │
│   │                                                                      │  │
│   │   • Properties are key-value pairs                                  │  │
│   │   • Keys are strings (convention: camelCase)                        │  │
│   │   • Values have specific types                                      │  │
│   │   • Properties are optional (schema-flexible)                       │  │
│   └─────────────────────────────────────────────────────────────────────┘  │
│                                                                             │
│   ┌─────────────────────────────────────────────────────────────────────┐  │
│   │                         INTERNAL ID                                  │  │
│   │                                                                      │  │
│   │   Neo4j assigns an internal ID to each node                         │  │
│   │   Access with: id(node) or elementId(node)                          │  │
│   │   ⚠️ Don't use for business logic - may be reused                   │  │
│   └─────────────────────────────────────────────────────────────────────┘  │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘
```

## 3.2 Creating Nodes with Cypher

### Basic Node Creation

```cypher
-- Create a simple node with one label
CREATE (p:Patient)
RETURN p

-- Create a node with properties
CREATE (p:Patient {
    id: 'P001',
    name: 'John Doe',
    dateOfBirth: date('1979-05-15'),
    gender: 'Male',
    active: true
})
RETURN p

-- Create a node with multiple labels
CREATE (p:Patient:VIP:Insured {
    id: 'P002',
    name: 'Jane Smith',
    tier: 'Gold'
})
RETURN p

-- Create multiple nodes at once
CREATE 
    (p1:Patient {id: 'P003', name: 'Alice Johnson'}),
    (p2:Patient {id: 'P004', name: 'Bob Wilson'}),
    (p3:Patient {id: 'P005', name: 'Carol Davis'})
RETURN p1, p2, p3
```

### Using MERGE for Idempotent Creation

MERGE is like "CREATE if not exists" - it only creates if the pattern doesn't exist.

```cypher
-- MERGE on unique property
MERGE (p:Patient {id: 'P001'})
ON CREATE SET 
    p.name = 'John Doe',
    p.createdAt = datetime()
ON MATCH SET 
    p.updatedAt = datetime()
RETURN p

-- MERGE with full properties
MERGE (d:Disease {id: 'E11'})
ON CREATE SET 
    d.name = 'Type 2 Diabetes Mellitus',
    d.category = 'Endocrine',
    d.icdCode = 'E11',
    d.createdAt = datetime()
RETURN d
```

### Property Types Examples

```cypher
CREATE (example:Example {
    -- String
    stringProp: 'Hello World',
    
    -- Numbers
    integerProp: 42,
    floatProp: 3.14159,
    
    -- Boolean
    booleanProp: true,
    
    -- Date and Time
    dateProp: date('2024-01-15'),
    timeProp: time('14:30:00'),
    datetimeProp: datetime('2024-01-15T14:30:00Z'),
    localDateTimeProp: localdatetime('2024-01-15T14:30:00'),
    
    -- Duration
    durationProp: duration('P1Y2M3DT4H5M6S'),
    
    -- Spatial
    pointProp: point({latitude: 40.7128, longitude: -74.0060}),
    
    -- Lists (must be homogeneous)
    stringListProp: ['a', 'b', 'c'],
    intListProp: [1, 2, 3, 4, 5]
})
RETURN example
```

## 3.3 Complete Python Implementation

Create `src/01_connection_basics.py`:

```python
"""
01_connection_basics.py
=======================
Neo4j Connection Basics and First Queries

This module demonstrates:
1. Connecting to Neo4j using different patterns
2. Basic queries and operations
3. Connection health checks
4. Session management

Run this file to test your Neo4j connection:
    python src/01_connection_basics.py
"""

import sys
import os
from typing import Dict, List, Any

# Add parent directory to path for imports
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

from src.utils.connection import Neo4jConnection, get_connection, get_driver


def demo_direct_connection():
    """
    Pattern 1: Direct connection management.
    
    Use this when you need explicit control over connection lifecycle.
    """
    print("\n📌 Pattern 1: Direct Connection")
    print("-" * 40)
    
    # Create connection
    conn = Neo4jConnection(
        uri="bolt://localhost:7687",
        username="neo4j",
        password="password123"
    )
    
    try:
        # Connect
        conn.connect()
        print("   Connected to Neo4j")
        
        # Execute query
        result = conn.execute_query("RETURN 'Hello from direct connection!' AS message")
        print(f"   Result: {result[0]['message']}")
        
    finally:
        # Always close
        conn.close()
        print("   Connection closed")


def demo_context_manager():
    """
    Pattern 2: Context manager (recommended for scripts).
    
    Automatically handles connection cleanup.
    """
    print("\n📌 Pattern 2: Context Manager")
    print("-" * 40)
    
    with Neo4jConnection() as conn:
        result = conn.execute_query("RETURN 'Hello from context manager!' AS message")
        print(f"   Result: {result[0]['message']}")
    
    print("   Connection automatically closed")


def demo_singleton():
    """
    Pattern 3: Singleton connection (recommended for applications).
    
    Reuses a single connection across the application.
    """
    print("\n📌 Pattern 3: Singleton Connection")
    print("-" * 40)
    
    # Get singleton (creates if needed)
    conn = get_connection()
    
    # Multiple queries reuse the same connection
    result1 = conn.execute_query("RETURN 1 AS num")
    result2 = conn.execute_query("RETURN 2 AS num")
    
    print(f"   Query 1: {result1[0]['num']}")
    print(f"   Query 2: {result2[0]['num']}")
    print("   Same connection reused for both queries")


def demo_driver_sessions():
    """
    Pattern 4: Direct driver and session usage.
    
    Use when you need fine-grained transaction control.
    """
    print("\n📌 Pattern 4: Driver and Sessions")
    print("-" * 40)
    
    driver = get_driver()
    
    # Explicit session management
    with driver.session() as session:
        # Read transaction
        result = session.run("MATCH (n) RETURN count(n) AS nodeCount")
        record = result.single()
        print(f"   Total nodes in database: {record['nodeCount']}")
    
    print("   Session closed automatically")


def demo_health_check():
    """
    Demonstrate health check functionality.
    """
    print("\n📌 Health Check")
    print("-" * 40)
    
    conn = get_connection()
    health = conn.health_check()
    
    for key, value in health.items():
        print(f"   {key}: {value}")


def demo_schema_inspection():
    """
    Demonstrate schema inspection.
    """
    print("\n📌 Schema Inspection")
    print("-" * 40)
    
    conn = get_connection()
    schema = conn.get_schema()
    
    print(f"   Labels: {schema['labels'][:5]}{'...' if len(schema['labels']) > 5 else ''}")
    print(f"   Relationship Types: {schema['relationship_types'][:5]}{'...' if len(schema['relationship_types']) > 5 else ''}")
    print(f"   Indexes: {len(schema['indexes'])}")
    print(f"   Constraints: {len(schema['constraints'])}")


# ============================================================
# MAIN EXECUTION
# ============================================================

if __name__ == "__main__":
    print("=" * 60)
    print("01: Neo4j Connection Basics")
    print("=" * 60)
    
    try:
        # Run all demonstrations
        demo_direct_connection()
        demo_context_manager()
        demo_singleton()
        demo_driver_sessions()
        demo_health_check()
        demo_schema_inspection()
        
        print("\n" + "=" * 60)
        print("✅ All connection patterns demonstrated successfully!")
        print("=" * 60)
        
    except Exception as e:
        print(f"\n❌ Error: {e}")
        print("\nMake sure Neo4j is running:")
        print("  docker-compose up -d")
```

Create `src/02_nodes_and_properties.py`:

```python
"""
02_nodes_and_properties.py
==========================
Creating and Managing Nodes in Neo4j

This module demonstrates:
1. Creating nodes with Cypher
2. Using dataclasses for type safety
3. Property types and validation
4. Bulk node creation
5. Node queries and updates

Healthcare Domain:
- Patient nodes with medical properties
- Disease nodes with ICD-11 codes
- Medication nodes with RxNorm codes
"""

import sys
import os
from dataclasses import dataclass, field
from datetime import date, datetime
from typing import List, Optional, Dict, Any
from enum import Enum

sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from src.utils.connection import get_driver


# ============================================================
# DATA MODELS
# ============================================================

class Gender(Enum):
    """Patient gender enumeration."""
    MALE = "Male"
    FEMALE = "Female"
    OTHER = "Other"
    UNKNOWN = "Unknown"


@dataclass
class Patient:
    """
    Patient entity model.
    
    Attributes:
        id: Unique patient identifier (required)
        name: Full name (required)
        date_of_birth: Birth date (required)
        gender: Patient gender
        blood_type: Blood type (A+, O-, etc.)
        allergies: List of known allergies
        email: Contact email
        phone: Contact phone
        active: Whether patient record is active
    """
    id: str
    name: str
    date_of_birth: date
    gender: Gender = Gender.UNKNOWN
    blood_type: Optional[str] = None
    allergies: List[str] = field(default_factory=list)
    email: Optional[str] = None
    phone: Optional[str] = None
    active: bool = True
    
    def to_dict(self) -> Dict[str, Any]:
        """Convert to dictionary for Cypher parameters."""
        return {
            "id": self.id,
            "name": self.name,
            "dateOfBirth": self.date_of_birth.isoformat(),
            "gender": self.gender.value,
            "bloodType": self.blood_type,
            "allergies": self.allergies,
            "email": self.email,
            "phone": self.phone,
            "active": self.active
        }


@dataclass
class Disease:
    """
    Disease entity model (ICD-11 based).
    
    Attributes:
        id: ICD-11 code (e.g., "E11" for Type 2 Diabetes)
        name: Disease name
        category: Disease category
        description: Detailed description
        severity: Default severity level
    """
    id: str  # ICD-11 code
    name: str
    category: str
    description: Optional[str] = None
    severity: str = "moderate"
    
    def to_dict(self) -> Dict[str, Any]:
        """Convert to dictionary for Cypher parameters."""
        return {
            "id": self.id,
            "name": self.name,
            "category": self.category,
            "description": self.description,
            "severity": self.severity
        }


@dataclass
class Medication:
    """
    Medication entity model (RxNorm based).
    
    Attributes:
        id: RxNorm CUI (e.g., "6809" for Metformin)
        name: Brand name
        generic_name: Generic name
        drug_class: Pharmacological class
        dosage_forms: Available forms (tablet, injection, etc.)
    """
    id: str  # RxNorm CUI
    name: str
    generic_name: str
    drug_class: str
    dosage_forms: List[str] = field(default_factory=list)
    
    def to_dict(self) -> Dict[str, Any]:
        """Convert to dictionary for Cypher parameters."""
        return {
            "id": self.id,
            "name": self.name,
            "genericName": self.generic_name,
            "drugClass": self.drug_class,
            "dosageForms": self.dosage_forms
        }


# ============================================================
# NODE MANAGER
# ============================================================

class NodeManager:
    """
    Manages node CRUD operations in Neo4j.
    
    Example usage:
        manager = NodeManager()
        
        # Create a patient
        patient = Patient(
            id="P001",
            name="John Doe",
            date_of_birth=date(1980, 5, 15),
            gender=Gender.MALE
        )
        created = manager.create_patient(patient)
        
        # Find patient
        found = manager.get_patient("P001")
        
        # Search patients
        patients = manager.search_patients(name_contains="John")
    """
    
    def __init__(self, driver=None):
        """Initialize with Neo4j driver."""
        self.driver = driver or get_driver()
    
    # =========================================================
    # PATIENT OPERATIONS
    # =========================================================
    
    def create_patient(self, patient: Patient) -> Dict[str, Any]:
        """
        Create a Patient node in Neo4j.
        
        Args:
            patient: Patient dataclass instance
            
        Returns:
            Created node properties
            
        Example:
            patient = Patient(id="P001", name="John", date_of_birth=date(1980,1,1))
            result = manager.create_patient(patient)
        """
        query = """
        CREATE (p:Patient {
            id: $id,
            name: $name,
            dateOfBirth: date($dateOfBirth),
            gender: $gender,
            bloodType: $bloodType,
            allergies: $allergies,
            email: $email,
            phone: $phone,
            active: $active,
            createdAt: datetime()
        })
        RETURN p
        """
        
        with self.driver.session() as session:
            result = session.run(query, **patient.to_dict())
            record = result.single()
            return dict(record["p"]) if record else None
    
    def create_patient_merge(self, patient: Patient) -> Dict[str, Any]:
        """
        Create or update Patient using MERGE (idempotent).
        
        Args:
            patient: Patient dataclass instance
            
        Returns:
            Node properties and whether it was created
        """
        query = """
        MERGE (p:Patient {id: $id})
        ON CREATE SET
            p.name = $name,
            p.dateOfBirth = date($dateOfBirth),
            p.gender = $gender,
            p.bloodType = $bloodType,
            p.allergies = $allergies,
            p.email = $email,
            p.phone = $phone,
            p.active = $active,
            p.createdAt = datetime()
        ON MATCH SET
            p.name = $name,
            p.gender = $gender,
            p.bloodType = $bloodType,
            p.allergies = $allergies,
            p.email = $email,
            p.phone = $phone,
            p.active = $active,
            p.updatedAt = datetime()
        RETURN p, 
               p.createdAt = p.updatedAt AS wasCreated
        """
        
        with self.driver.session() as session:
            result = session.run(query, **patient.to_dict())
            record = result.single()
            if record:
                return {
                    "node": dict(record["p"]),
                    "was_created": record["wasCreated"]
                }
            return None
    
    def bulk_create_patients(
        self, 
        patients: List[Patient], 
        batch_size: int = 1000
    ) -> int:
        """
        Bulk create patients using UNWIND for efficiency.
        
        Args:
            patients: List of Patient objects
            batch_size: Number of patients per batch
            
        Returns:
            Total number of patients created
            
        Note:
            UNWIND converts a list into rows, enabling batch operations.
            This is much faster than individual CREATE statements.
        """
        query = """
        UNWIND $patients AS p
        CREATE (patient:Patient {
            id: p.id,
            name: p.name,
            dateOfBirth: date(p.dateOfBirth),
            gender: p.gender,
            bloodType: p.bloodType,
            allergies: p.allergies,
            active: p.active,
            createdAt: datetime()
        })
        RETURN count(*) AS created
        """
        
        # Convert to list of dicts
        patient_data = [p.to_dict() for p in patients]
        
        total_created = 0
        with self.driver.session() as session:
            # Process in batches
            for i in range(0, len(patient_data), batch_size):
                batch = patient_data[i:i + batch_size]
                result = session.run(query, patients=batch)
                total_created += result.single()["created"]
        
        return total_created
    
    def get_patient(self, patient_id: str) -> Optional[Dict[str, Any]]:
        """
        Get a patient by ID.
        
        Args:
            patient_id: Patient ID
            
        Returns:
            Patient properties or None if not found
        """
        query = """
        MATCH (p:Patient {id: $id})
        RETURN p
        """
        
        with self.driver.session() as session:
            result = session.run(query, id=patient_id)
            record = result.single()
            return dict(record["p"]) if record else None
    
    def search_patients(
        self,
        name_contains: str = None,
        gender: str = None,
        min_age: int = None,
        max_age: int = None,
        active_only: bool = True,
        limit: int = 100
    ) -> List[Dict[str, Any]]:
        """
        Search patients with multiple filters.
        
        Args:
            name_contains: Filter by name substring
            gender: Filter by gender
            min_age: Minimum age
            max_age: Maximum age
            active_only: Only return active patients
            limit: Maximum results
            
        Returns:
            List of matching patients
        """
        conditions = []
        params = {"limit": limit}
        
        if name_contains:
            conditions.append("toLower(p.name) CONTAINS toLower($name)")
            params["name"] = name_contains
        
        if gender:
            conditions.append("p.gender = $gender")
            params["gender"] = gender
        
        if min_age is not None:
            conditions.append("duration.between(p.dateOfBirth, date()).years >= $minAge")
            params["minAge"] = min_age
        
        if max_age is not None:
            conditions.append("duration.between(p.dateOfBirth, date()).years <= $maxAge")
            params["maxAge"] = max_age
        
        if active_only:
            conditions.append("p.active = true")
        
        where_clause = "WHERE " + " AND ".join(conditions) if conditions else ""
        
        query = f"""
        MATCH (p:Patient)
        {where_clause}
        RETURN p
        ORDER BY p.name
        LIMIT $limit
        """
        
        with self.driver.session() as session:
            result = session.run(query, **params)
            return [dict(r["p"]) for r in result]
    
    def update_patient(
        self, 
        patient_id: str, 
        properties: Dict[str, Any]
    ) -> Optional[Dict[str, Any]]:
        """
        Update patient properties.
        
        Args:
            patient_id: Patient ID
            properties: Properties to update
            
        Returns:
            Updated patient or None
        """
        set_clauses = [f"p.{k} = ${k}" for k in properties.keys()]
        set_clauses.append("p.updatedAt = datetime()")
        set_str = ", ".join(set_clauses)
        
        query = f"""
        MATCH (p:Patient {{id: $id}})
        SET {set_str}
        RETURN p
        """
        
        params = {"id": patient_id, **properties}
        
        with self.driver.session() as session:
            result = session.run(query, **params)
            record = result.single()
            return dict(record["p"]) if record else None
    
    def delete_patient(self, patient_id: str, detach: bool = True) -> bool:
        """
        Delete a patient.
        
        Args:
            patient_id: Patient ID
            detach: Also delete relationships
            
        Returns:
            True if deleted, False if not found
        """
        delete_cmd = "DETACH DELETE" if detach else "DELETE"
        
        query = f"""
        MATCH (p:Patient {{id: $id}})
        {delete_cmd} p
        RETURN count(*) AS deleted
        """
        
        with self.driver.session() as session:
            result = session.run(query, id=patient_id)
            return result.single()["deleted"] > 0
    
    # =========================================================
    # DISEASE OPERATIONS
    # =========================================================
    
    def create_disease(self, disease: Disease) -> Dict[str, Any]:
        """Create a Disease node."""
        query = """
        MERGE (d:Disease {id: $id})
        ON CREATE SET
            d.name = $name,
            d.category = $category,
            d.description = $description,
            d.severity = $severity,
            d.createdAt = datetime()
        ON MATCH SET
            d.updatedAt = datetime()
        RETURN d
        """
        
        with self.driver.session() as session:
            result = session.run(query, **disease.to_dict())
            record = result.single()
            return dict(record["d"]) if record else None
    
    # =========================================================
    # MEDICATION OPERATIONS
    # =========================================================
    
    def create_medication(self, medication: Medication) -> Dict[str, Any]:
        """Create a Medication node."""
        query = """
        MERGE (m:Medication {id: $id})
        ON CREATE SET
            m.name = $name,
            m.genericName = $genericName,
            m.drugClass = $drugClass,
            m.dosageForms = $dosageForms,
            m.createdAt = datetime()
        ON MATCH SET
            m.updatedAt = datetime()
        RETURN m
        """
        
        with self.driver.session() as session:
            result = session.run(query, **medication.to_dict())
            record = result.single()
            return dict(record["m"]) if record else None
    
    # =========================================================
    # UTILITY METHODS
    # =========================================================
    
    def count_nodes(self, label: str) -> int:
        """Count nodes with a specific label."""
        query = f"MATCH (n:{label}) RETURN count(n) AS count"
        
        with self.driver.session() as session:
            result = session.run(query)
            return result.single()["count"]
    
    def delete_all_by_label(self, label: str) -> int:
        """Delete all nodes with a specific label."""
        query = f"MATCH (n:{label}) DETACH DELETE n RETURN count(*) AS deleted"
        
        with self.driver.session() as session:
            result = session.run(query)
            return result.single()["deleted"]


# ============================================================
# DEMONSTRATION
# ============================================================

if __name__ == "__main__":
    print("=" * 60)
    print("02: Nodes and Properties")
    print("=" * 60)
    
    manager = NodeManager()
    
    # Clean up any existing test data
    print("\n🧹 Cleaning up test data...")
    manager.delete_all_by_label("Patient")
    manager.delete_all_by_label("Disease")
    manager.delete_all_by_label("Medication")
    
    # Create patients
    print("\n📌 Creating Patients")
    print("-" * 40)
    
    patients = [
        Patient(
            id="P001",
            name="John Doe",
            date_of_birth=date(1980, 5, 15),
            gender=Gender.MALE,
            blood_type="O+",
            allergies=["Penicillin"]
        ),
        Patient(
            id="P002",
            name="Jane Smith",
            date_of_birth=date(1992, 8, 22),
            gender=Gender.FEMALE,
            blood_type="A-"
        ),
        Patient(
            id="P003",
            name="Bob Wilson",
            date_of_birth=date(1975, 12, 1),
            gender=Gender.MALE,
            blood_type="B+",
            allergies=["Latex", "Sulfa"]
        )
    ]
    
    for patient in patients:
        result = manager.create_patient(patient)
        print(f"   Created: {result['name']} ({result['id']})")
    
    # Create diseases
    print("\n📌 Creating Diseases")
    print("-" * 40)
    
    diseases = [
        Disease(
            id="E11",
            name="Type 2 Diabetes Mellitus",
            category="Endocrine",
            description="A chronic metabolic disorder"
        ),
        Disease(
            id="I10",
            name="Essential Hypertension",
            category="Cardiovascular",
            description="High blood pressure"
        ),
        Disease(
            id="J45",
            name="Asthma",
            category="Respiratory",
            description="Chronic inflammatory disease of airways"
        )
    ]
    
    for disease in diseases:
        result = manager.create_disease(disease)
        print(f"   Created: {result['name']} ({result['id']})")
    
    # Create medications
    print("\n📌 Creating Medications")
    print("-" * 40)
    
    medications = [
        Medication(
            id="6809",
            name="Metformin",
            generic_name="Metformin Hydrochloride",
            drug_class="Biguanide",
            dosage_forms=["Tablet", "Extended-release tablet"]
        ),
        Medication(
            id="29046",
            name="Lisinopril",
            generic_name="Lisinopril",
            drug_class="ACE Inhibitor",
            dosage_forms=["Tablet"]
        )
    ]
    
    for medication in medications:
        result = manager.create_medication(medication)
        print(f"   Created: {result['name']} ({result['id']})")
    
    # Search and query
    print("\n📌 Querying Nodes")
    print("-" * 40)
    
    # Find specific patient
    john = manager.get_patient("P001")
    print(f"   Found: {john['name']}, DOB: {john['dateOfBirth']}")
    
    # Search patients
    male_patients = manager.search_patients(gender="Male")
    print(f"   Male patients: {len(male_patients)}")
    
    # Update patient
    print("\n📌 Updating Patient")
    print("-" * 40)
    
    updated = manager.update_patient("P001", {"email": "john.doe@email.com"})
    print(f"   Updated: {updated['name']}, email: {updated['email']}")
    
    # Count nodes
    print("\n📌 Node Counts")
    print("-" * 40)
    print(f"   Patients: {manager.count_nodes('Patient')}")
    print(f"   Diseases: {manager.count_nodes('Disease')}")
    print(f"   Medications: {manager.count_nodes('Medication')}")
    
    print("\n" + "=" * 60)
    print("✅ Node operations completed successfully!")
    print("=" * 60)
```

---

# 4. Relationships and Patterns

## 4.1 Understanding Relationships

Relationships are first-class citizens in Neo4j, representing the connections between entities.

### Relationship Anatomy

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                         RELATIONSHIP STRUCTURE                              │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│   START NODE                 RELATIONSHIP                    END NODE       │
│   ──────────                 ────────────                    ────────       │
│                                                                             │
│   ┌──────────┐    ────[DIAGNOSED_WITH]────►    ┌──────────┐               │
│   │ Patient  │                                  │ Disease  │               │
│   │ (P001)   │    Properties:                   │  (E11)   │               │
│   └──────────┘    • diagnosedDate: date()      └──────────┘               │
│                   • diagnosedBy: "Dr. Smith"                               │
│                   • status: "active"                                        │
│                   • severity: "moderate"                                    │
│                                                                             │
│   KEY CHARACTERISTICS:                                                      │
│   ────────────────────                                                      │
│   • ALWAYS has direction (even if queried both ways)                       │
│   • ALWAYS has exactly one type                                            │
│   • CAN have properties (optional)                                         │
│   • Convention: SCREAMING_SNAKE_CASE for types                             │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘
```

### Healthcare Relationship Types

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                    HEALTHCARE RELATIONSHIP MODEL                            │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│                         ┌───────────────┐                                   │
│                         │    Doctor     │                                   │
│                         └───────┬───────┘                                   │
│                                 │                                           │
│                           TREATED_BY                                        │
│                                 │                                           │
│   ┌───────────────┐      ┌─────▼─────┐      ┌───────────────┐             │
│   │  Medication   │◄─TAKES─│  Patient  │─DIAGNOSED_WITH─►│   Disease    │ │
│   │   (RxNorm)    │        │           │                   │   (ICD-11)   │ │
│   └───────┬───────┘        └─────┬─────┘                   └───────┬───────┘│
│           │                      │                                 │        │
│        TREATS                ALLERGIC_TO                       TREATS       │
│           │                      │                                 │        │
│           │                      ▼                                 │        │
│           │               ┌───────────────┐                        │        │
│           │               │   Substance   │                        │        │
│           │               └───────────────┘                        │        │
│           │                                                        │        │
│           └────────────────────────┬───────────────────────────────┘        │
│                                    │                                        │
│                             INTERACTS_WITH                                  │
│                                    │                                        │
│                            ┌───────▼───────┐                               │
│                            │  Medication   │                               │
│                            └───────────────┘                               │
│                                                                             │
│   RELATIONSHIP PROPERTIES:                                                  │
│   ─────────────────────────                                                 │
│   DIAGNOSED_WITH: diagnosedDate, diagnosedBy, status, severity, notes      │
│   TAKES: dosage, frequency, prescribedDate, prescribedBy, status, refills  │
│   TREATS: effectiveness, firstLine, evidenceLevel                          │
│   INTERACTS_WITH: severity, description, mechanism                         │
│   ALLERGIC_TO: reactionType, severity, discoveredDate                      │
│   TREATED_BY: since, specialty, isPrimary                                  │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘
```

## 4.2 Creating Relationships with Cypher

```cypher
-- Basic relationship creation
MATCH (p:Patient {id: 'P001'})
MATCH (d:Disease {id: 'E11'})
CREATE (p)-[:DIAGNOSED_WITH]->(d)
RETURN p, d

-- Relationship with properties
MATCH (p:Patient {id: 'P001'})
MATCH (d:Disease {id: 'E11'})
CREATE (p)-[r:DIAGNOSED_WITH {
    diagnosedDate: date('2024-01-15'),
    diagnosedBy: 'Dr. Sarah Johnson',
    status: 'active',
    severity: 'moderate',
    notes: 'Initial diagnosis following routine screening'
}]->(d)
RETURN p, r, d

-- MERGE relationship (create if not exists)
MATCH (p:Patient {id: 'P001'})
MATCH (m:Medication {id: '6809'})
MERGE (p)-[r:TAKES]->(m)
ON CREATE SET
    r.prescribedDate = date(),
    r.dosage = '500mg',
    r.frequency = 'twice daily',
    r.status = 'active',
    r.createdAt = datetime()
ON MATCH SET
    r.lastVerified = datetime()
RETURN p, r, m

-- Create node and relationship together
CREATE (p:Patient {id: 'P100', name: 'New Patient'})
CREATE (d:Disease {id: 'E11', name: 'Type 2 Diabetes'})
CREATE (p)-[:DIAGNOSED_WITH {diagnosedDate: date()}]->(d)
RETURN p, d
```

## 4.3 Python Relationship Implementation

Create `src/03_relationships.py`:

```python
"""
03_relationships.py
===================
Creating and Managing Relationships in Neo4j

This module demonstrates:
1. Relationship creation patterns
2. Relationship properties
3. Querying connected data
4. Updating and deleting relationships
5. Path queries

Healthcare relationships modeled:
- DIAGNOSED_WITH (Patient → Disease)
- TAKES (Patient → Medication)
- TREATS (Medication → Disease)
- INTERACTS_WITH (Medication ↔ Medication)
- ALLERGIC_TO (Patient → Substance)
"""

import sys
import os
from dataclasses import dataclass, field
from datetime import date, datetime
from typing import List, Optional, Dict, Any
from enum import Enum

sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from src.utils.connection import get_driver


# ============================================================
# RELATIONSHIP TYPE DEFINITIONS
# ============================================================

class RelationshipType(Enum):
    """Standard relationship types in healthcare domain."""
    DIAGNOSED_WITH = "DIAGNOSED_WITH"
    TAKES = "TAKES"
    TREATS = "TREATS"
    INTERACTS_WITH = "INTERACTS_WITH"
    ALLERGIC_TO = "ALLERGIC_TO"
    TREATED_BY = "TREATED_BY"
    PART_OF = "PART_OF"
    MENTIONS = "MENTIONS"


# ============================================================
# RELATIONSHIP PROPERTY MODELS
# ============================================================

@dataclass
class Diagnosis:
    """Properties for DIAGNOSED_WITH relationship."""
    diagnosed_date: date
    diagnosed_by: str
    status: str = "active"  # active, resolved, remission
    severity: str = "moderate"  # mild, moderate, severe
    notes: Optional[str] = None
    
    def to_dict(self) -> Dict[str, Any]:
        return {
            "diagnosedDate": self.diagnosed_date.isoformat(),
            "diagnosedBy": self.diagnosed_by,
            "status": self.status,
            "severity": self.severity,
            "notes": self.notes
        }


@dataclass
class Prescription:
    """Properties for TAKES relationship."""
    prescribed_date: date
    prescribed_by: str
    dosage: str
    frequency: str  # "once daily", "twice daily", "as needed"
    duration_days: Optional[int] = None
    refills: int = 0
    status: str = "active"  # active, discontinued, completed
    reason: Optional[str] = None
    
    def to_dict(self) -> Dict[str, Any]:
        return {
            "prescribedDate": self.prescribed_date.isoformat(),
            "prescribedBy": self.prescribed_by,
            "dosage": self.dosage,
            "frequency": self.frequency,
            "durationDays": self.duration_days,
            "refills": self.refills,
            "status": self.status,
            "reason": self.reason
        }


@dataclass
class Treatment:
    """Properties for TREATS relationship."""
    effectiveness: str = "moderate"  # low, moderate, high
    first_line: bool = False
    evidence_level: str = "B"  # A, B, C (highest to lowest)
    
    def to_dict(self) -> Dict[str, Any]:
        return {
            "effectiveness": self.effectiveness,
            "firstLine": self.first_line,
            "evidenceLevel": self.evidence_level
        }


@dataclass
class DrugInteraction:
    """Properties for INTERACTS_WITH relationship."""
    severity: str  # mild, moderate, severe, contraindicated
    description: str
    mechanism: Optional[str] = None
    
    def to_dict(self) -> Dict[str, Any]:
        return {
            "severity": self.severity,
            "description": self.description,
            "mechanism": self.mechanism
        }


# ============================================================
# RELATIONSHIP MANAGER
# ============================================================

class RelationshipManager:
    """
    Manages relationship operations in Neo4j.
    
    Example:
        manager = RelationshipManager()
        
        # Create diagnosis
        diagnosis = Diagnosis(
            diagnosed_date=date(2024, 1, 15),
            diagnosed_by="Dr. Smith",
            severity="moderate"
        )
        manager.create_diagnosis("P001", "E11", diagnosis)
        
        # Get patient's conditions
        conditions = manager.get_patient_diagnoses("P001")
    """
    
    def __init__(self, driver=None):
        self.driver = driver or get_driver()
    
    # =========================================================
    # DIAGNOSIS RELATIONSHIPS
    # =========================================================
    
    def create_diagnosis(
        self,
        patient_id: str,
        disease_id: str,
        diagnosis: Diagnosis
    ) -> Dict[str, Any]:
        """
        Create a DIAGNOSED_WITH relationship.
        
        Args:
            patient_id: Patient ID
            disease_id: Disease ID (ICD-11 code)
            diagnosis: Diagnosis properties
            
        Returns:
            Created relationship details
        """
        query = """
        MATCH (p:Patient {id: $patientId})
        MATCH (d:Disease {id: $diseaseId})
        CREATE (p)-[r:DIAGNOSED_WITH {
            diagnosedDate: date($diagnosedDate),
            diagnosedBy: $diagnosedBy,
            status: $status,
            severity: $severity,
            notes: $notes,
            createdAt: datetime()
        }]->(d)
        RETURN 
            p.name AS patient,
            d.name AS disease,
            properties(r) AS diagnosis
        """
        
        params = {
            "patientId": patient_id,
            "diseaseId": disease_id,
            **diagnosis.to_dict()
        }
        
        with self.driver.session() as session:
            result = session.run(query, **params)
            record = result.single()
            return record.data() if record else None
    
    def get_patient_diagnoses(
        self, 
        patient_id: str,
        active_only: bool = False
    ) -> List[Dict[str, Any]]:
        """
        Get all diagnoses for a patient.
        
        Args:
            patient_id: Patient ID
            active_only: Only return active diagnoses
            
        Returns:
            List of diagnoses with disease information
        """
        status_filter = "AND r.status = 'active'" if active_only else ""
        
        query = f"""
        MATCH (p:Patient {{id: $id}})-[r:DIAGNOSED_WITH]->(d:Disease)
        WHERE true {status_filter}
        RETURN 
            d.id AS diseaseId,
            d.name AS disease,
            d.category AS category,
            r.diagnosedDate AS diagnosedDate,
            r.diagnosedBy AS diagnosedBy,
            r.status AS status,
            r.severity AS severity
        ORDER BY r.diagnosedDate DESC
        """
        
        with self.driver.session() as session:
            result = session.run(query, id=patient_id)
            return [r.data() for r in result]
    
    def update_diagnosis_status(
        self,
        patient_id: str,
        disease_id: str,
        new_status: str,
        notes: str = None
    ) -> bool:
        """
        Update the status of a diagnosis.
        
        Args:
            patient_id: Patient ID
            disease_id: Disease ID
            new_status: New status (active, resolved, remission)
            notes: Optional notes about the change
            
        Returns:
            True if updated
        """
        query = """
        MATCH (p:Patient {id: $patientId})-[r:DIAGNOSED_WITH]->(d:Disease {id: $diseaseId})
        SET r.status = $status,
            r.updatedAt = datetime()
        """
        
        if notes:
            query += ", r.notes = COALESCE(r.notes, '') + '\\n' + $notes"
        
        query += " RETURN r"
        
        with self.driver.session() as session:
            result = session.run(
                query,
                patientId=patient_id,
                diseaseId=disease_id,
                status=new_status,
                notes=notes
            )
            return result.single() is not None
    
    # =========================================================
    # PRESCRIPTION RELATIONSHIPS
    # =========================================================
    
    def create_prescription(
        self,
        patient_id: str,
        medication_id: str,
        prescription: Prescription
    ) -> Dict[str, Any]:
        """
        Create a TAKES relationship (prescription).
        
        Args:
            patient_id: Patient ID
            medication_id: Medication ID (RxNorm CUI)
            prescription: Prescription properties
            
        Returns:
            Created relationship details
        """
        query = """
        MATCH (p:Patient {id: $patientId})
        MATCH (m:Medication {id: $medicationId})
        CREATE (p)-[r:TAKES {
            prescribedDate: date($prescribedDate),
            prescribedBy: $prescribedBy,
            dosage: $dosage,
            frequency: $frequency,
            durationDays: $durationDays,
            refills: $refills,
            status: $status,
            reason: $reason,
            createdAt: datetime()
        }]->(m)
        RETURN 
            p.name AS patient,
            m.name AS medication,
            m.drugClass AS drugClass,
            properties(r) AS prescription
        """
        
        params = {
            "patientId": patient_id,
            "medicationId": medication_id,
            **prescription.to_dict()
        }
        
        with self.driver.session() as session:
            result = session.run(query, **params)
            record = result.single()
            return record.data() if record else None
    
    def get_patient_medications(
        self,
        patient_id: str,
        active_only: bool = True
    ) -> List[Dict[str, Any]]:
        """
        Get all medications for a patient.
        
        Args:
            patient_id: Patient ID
            active_only: Only return active prescriptions
            
        Returns:
            List of medications with prescription details
        """
        status_filter = "AND r.status = 'active'" if active_only else ""
        
        query = f"""
        MATCH (p:Patient {{id: $id}})-[r:TAKES]->(m:Medication)
        WHERE true {status_filter}
        RETURN 
            m.id AS medicationId,
            m.name AS medication,
            m.genericName AS genericName,
            m.drugClass AS drugClass,
            r.dosage AS dosage,
            r.frequency AS frequency,
            r.prescribedDate AS prescribedDate,
            r.prescribedBy AS prescribedBy,
            r.status AS status,
            r.refills AS refillsRemaining
        ORDER BY r.prescribedDate DESC
        """
        
        with self.driver.session() as session:
            result = session.run(query, id=patient_id)
            return [r.data() for r in result]
    
    def refill_prescription(
        self,
        patient_id: str,
        medication_id: str
    ) -> Dict[str, Any]:
        """
        Process a prescription refill.
        
        Decrements refills count and logs the refill.
        """
        query = """
        MATCH (p:Patient {id: $patientId})-[r:TAKES]->(m:Medication {id: $medicationId})
        WHERE r.status = 'active' AND r.refills > 0
        SET r.refills = r.refills - 1,
            r.lastRefillDate = date(),
            r.updatedAt = datetime()
        RETURN 
            m.name AS medication,
            r.refills AS refillsRemaining,
            r.lastRefillDate AS lastRefill
        """
        
        with self.driver.session() as session:
            result = session.run(
                query,
                patientId=patient_id,
                medicationId=medication_id
            )
            record = result.single()
            return record.data() if record else None
    
    def discontinue_medication(
        self,
        patient_id: str,
        medication_id: str,
        reason: str
    ) -> bool:
        """
        Discontinue a medication.
        """
        query = """
        MATCH (p:Patient {id: $patientId})-[r:TAKES]->(m:Medication {id: $medicationId})
        WHERE r.status = 'active'
        SET r.status = 'discontinued',
            r.discontinuedDate = date(),
            r.discontinuedReason = $reason,
            r.updatedAt = datetime()
        RETURN r
        """
        
        with self.driver.session() as session:
            result = session.run(
                query,
                patientId=patient_id,
                medicationId=medication_id,
                reason=reason
            )
            return result.single() is not None
    
    # =========================================================
    # TREATMENT RELATIONSHIPS
    # =========================================================
    
    def create_treatment(
        self,
        medication_id: str,
        disease_id: str,
        treatment: Treatment
    ) -> Dict[str, Any]:
        """
        Create a TREATS relationship.
        """
        query = """
        MATCH (m:Medication {id: $medicationId})
        MATCH (d:Disease {id: $diseaseId})
        MERGE (m)-[r:TREATS]->(d)
        ON CREATE SET
            r.effectiveness = $effectiveness,
            r.firstLine = $firstLine,
            r.evidenceLevel = $evidenceLevel,
            r.createdAt = datetime()
        RETURN 
            m.name AS medication,
            d.name AS disease,
            properties(r) AS treatment
        """
        
        params = {
            "medicationId": medication_id,
            "diseaseId": disease_id,
            **treatment.to_dict()
        }
        
        with self.driver.session() as session:
            result = session.run(query, **params)
            record = result.single()
            return record.data() if record else None
    
    def get_disease_treatments(
        self,
        disease_id: str,
        first_line_only: bool = False
    ) -> List[Dict[str, Any]]:
        """
        Get all medications that treat a disease.
        """
        first_line_filter = "AND r.firstLine = true" if first_line_only else ""
        
        query = f"""
        MATCH (m:Medication)-[r:TREATS]->(d:Disease {{id: $id}})
        WHERE true {first_line_filter}
        RETURN 
            m.id AS medicationId,
            m.name AS medication,
            m.drugClass AS drugClass,
            r.effectiveness AS effectiveness,
            r.firstLine AS firstLine,
            r.evidenceLevel AS evidenceLevel
        ORDER BY r.firstLine DESC, r.effectiveness DESC
        """
        
        with self.driver.session() as session:
            result = session.run(query, id=disease_id)
            return [r.data() for r in result]
    
    # =========================================================
    # DRUG INTERACTION RELATIONSHIPS
    # =========================================================
    
    def create_drug_interaction(
        self,
        medication1_id: str,
        medication2_id: str,
        interaction: DrugInteraction
    ) -> Dict[str, Any]:
        """
        Create an INTERACTS_WITH relationship between medications.
        
        Note: This creates a bidirectional relationship pattern.
        """
        query = """
        MATCH (m1:Medication {id: $med1Id})
        MATCH (m2:Medication {id: $med2Id})
        WHERE m1.id < m2.id  // Ensure only one relationship per pair
        MERGE (m1)-[r:INTERACTS_WITH]-(m2)
        ON CREATE SET
            r.severity = $severity,
            r.description = $description,
            r.mechanism = $mechanism,
            r.createdAt = datetime()
        RETURN 
            m1.name AS medication1,
            m2.name AS medication2,
            properties(r) AS interaction
        """
        
        params = {
            "med1Id": medication1_id,
            "med2Id": medication2_id,
            **interaction.to_dict()
        }
        
        with self.driver.session() as session:
            result = session.run(query, **params)
            record = result.single()
            return record.data() if record else None
    
    def check_drug_interactions(
        self,
        patient_id: str
    ) -> List[Dict[str, Any]]:
        """
        Check for drug interactions in a patient's active medications.
        
        This is a critical safety check!
        """
        query = """
        MATCH (p:Patient {id: $id})-[t1:TAKES]->(m1:Medication)
        MATCH (p)-[t2:TAKES]->(m2:Medication)
        WHERE t1.status = 'active' 
          AND t2.status = 'active'
          AND m1.id < m2.id
        OPTIONAL MATCH (m1)-[i:INTERACTS_WITH]-(m2)
        WHERE i IS NOT NULL
        RETURN 
            m1.name AS drug1,
            m2.name AS drug2,
            i.severity AS severity,
            i.description AS description,
            i.mechanism AS mechanism
        ORDER BY 
            CASE i.severity
                WHEN 'contraindicated' THEN 1
                WHEN 'severe' THEN 2
                WHEN 'moderate' THEN 3
                WHEN 'mild' THEN 4
                ELSE 5
            END
        """
        
        with self.driver.session() as session:
            result = session.run(query, id=patient_id)
            return [r.data() for r in result if r["severity"] is not None]
    
    # =========================================================
    # PATH QUERIES
    # =========================================================
    
    def get_treatment_path(
        self,
        patient_id: str
    ) -> List[Dict[str, Any]]:
        """
        Get the complete treatment path for a patient.
        
        Shows: Patient → Disease → Available Medications
        """
        query = """
        MATCH (p:Patient {id: $id})-[d:DIAGNOSED_WITH]->(disease:Disease)
        OPTIONAL MATCH (med:Medication)-[t:TREATS]->(disease)
        RETURN 
            p.name AS patient,
            disease.name AS condition,
            d.status AS diagnosisStatus,
            d.severity AS severity,
            collect({
                medication: med.name,
                drugClass: med.drugClass,
                effectiveness: t.effectiveness,
                firstLine: t.firstLine
            }) AS treatmentOptions
        ORDER BY d.diagnosedDate DESC
        """
        
        with self.driver.session() as session:
            result = session.run(query, id=patient_id)
            return [r.data() for r in result]
    
    # =========================================================
    # RELATIONSHIP DELETION
    # =========================================================
    
    def delete_relationship(
        self,
        from_label: str,
        from_id: str,
        rel_type: str,
        to_label: str,
        to_id: str
    ) -> bool:
        """
        Delete a specific relationship.
        """
        query = f"""
        MATCH (a:{from_label} {{id: $fromId}})-[r:{rel_type}]->(b:{to_label} {{id: $toId}})
        DELETE r
        RETURN count(*) AS deleted
        """
        
        with self.driver.session() as session:
            result = session.run(
                query,
                fromId=from_id,
                toId=to_id
            )
            return result.single()["deleted"] > 0


# ============================================================
# DEMONSTRATION
# ============================================================

if __name__ == "__main__":
    print("=" * 60)
    print("03: Relationships")
    print("=" * 60)
    
    manager = RelationshipManager()
    
    # Create diagnosis
    print("\n📌 Creating Diagnoses")
    print("-" * 40)
    
    diagnosis = Diagnosis(
        diagnosed_date=date(2024, 1, 15),
        diagnosed_by="Dr. Sarah Johnson",
        status="active",
        severity="moderate"
    )
    
    result = manager.create_diagnosis("P001", "E11", diagnosis)
    if result:
        print(f"   {result['patient']} diagnosed with {result['disease']}")
    
    # Create prescription
    print("\n📌 Creating Prescriptions")
    print("-" * 40)
    
    prescription = Prescription(
        prescribed_date=date(2024, 1, 15),
        prescribed_by="Dr. Sarah Johnson",
        dosage="500mg",
        frequency="twice daily",
        duration_days=90,
        refills=3
    )
    
    result = manager.create_prescription("P001", "6809", prescription)
    if result:
        print(f"   {result['patient']} prescribed {result['medication']}")
    
    # Create treatment relationship
    print("\n📌 Creating Treatment Relationships")
    print("-" * 40)
    
    treatment = Treatment(
        effectiveness="high",
        first_line=True,
        evidence_level="A"
    )
    
    result = manager.create_treatment("6809", "E11", treatment)
    if result:
        print(f"   {result['medication']} treats {result['disease']}")
    
    # Query relationships
    print("\n📌 Querying Relationships")
    print("-" * 40)
    
    diagnoses = manager.get_patient_diagnoses("P001")
    print(f"   Patient P001 has {len(diagnoses)} diagnosis(es)")
    
    medications = manager.get_patient_medications("P001")
    print(f"   Patient P001 takes {len(medications)} medication(s)")
    
    treatments = manager.get_disease_treatments("E11")
    print(f"   Disease E11 has {len(treatments)} treatment option(s)")
    
    # Treatment path
    print("\n📌 Treatment Path Analysis")
    print("-" * 40)
    
    paths = manager.get_treatment_path("P001")
    for path in paths:
        print(f"   Condition: {path['condition']}")
        print(f"   Status: {path['diagnosisStatus']}, Severity: {path['severity']}")
        if path['treatmentOptions']:
            for opt in path['treatmentOptions']:
                if opt['medication']:
                    print(f"     → {opt['medication']} ({opt['effectiveness']} effectiveness)")
    
    print("\n" + "=" * 60)
    print("✅ Relationship operations completed successfully!")
    print("=" * 60)
```

---

*[Tutorial continues with sections 5-30...]*


---

# 5. Basic CRUD Operations

## 5.1 CREATE Operations

### Single Node Creation

```cypher
-- Create a single node
CREATE (p:Patient {
    id: 'P001',
    name: 'John Doe',
    dateOfBirth: date('1980-05-15'),
    gender: 'Male',
    active: true
})
RETURN p

-- Create with multiple labels
CREATE (p:Patient:VIP {
    id: 'P002',
    name: 'Jane Smith',
    memberTier: 'Gold'
})
RETURN p
```

### Multiple Nodes

```cypher
-- Create multiple nodes in one query
CREATE 
    (p1:Patient {id: 'P010', name: 'Alice'}),
    (p2:Patient {id: 'P011', name: 'Bob'}),
    (p3:Patient {id: 'P012', name: 'Carol'})
RETURN p1, p2, p3
```

### CREATE with Relationships

```cypher
-- Create nodes and relationships together
CREATE (p:Patient {id: 'P020', name: 'David'})
CREATE (d:Disease {id: 'E11', name: 'Diabetes'})
CREATE (p)-[:DIAGNOSED_WITH {date: date()}]->(d)
RETURN p, d
```

### MERGE (Idempotent Create)

```cypher
-- MERGE only creates if the pattern doesn't exist
MERGE (p:Patient {id: 'P001'})
ON CREATE SET 
    p.name = 'John Doe',
    p.createdAt = datetime()
ON MATCH SET 
    p.lastAccessedAt = datetime()
RETURN p, p.createdAt = p.lastAccessedAt AS wasCreated
```

## 5.2 READ Operations

### Basic MATCH

```cypher
-- Match all nodes of a label
MATCH (p:Patient)
RETURN p
LIMIT 100

-- Match by property
MATCH (p:Patient {id: 'P001'})
RETURN p

-- Match with property access
MATCH (p:Patient {id: 'P001'})
RETURN p.name AS name, p.dateOfBirth AS dob
```

### WHERE Clauses

```cypher
-- Simple comparison
MATCH (p:Patient)
WHERE p.active = true
RETURN p

-- Multiple conditions
MATCH (p:Patient)
WHERE p.active = true 
  AND p.gender = 'Male'
  AND p.dateOfBirth > date('1980-01-01')
RETURN p

-- String operations
MATCH (p:Patient)
WHERE p.name STARTS WITH 'John'
   OR p.name CONTAINS 'Smith'
   OR toLower(p.name) ENDS WITH 'doe'
RETURN p

-- List operations
MATCH (p:Patient)
WHERE 'Penicillin' IN p.allergies
RETURN p

-- NULL checks
MATCH (p:Patient)
WHERE p.email IS NOT NULL
RETURN p

-- Pattern predicates
MATCH (p:Patient)
WHERE EXISTS {
    MATCH (p)-[:DIAGNOSED_WITH]->(:Disease {category: 'Cardiovascular'})
}
RETURN p
```

### Pattern Matching

```cypher
-- Match relationships
MATCH (p:Patient)-[r:DIAGNOSED_WITH]->(d:Disease)
WHERE p.id = 'P001'
RETURN p.name, d.name, r.diagnosedDate

-- Optional match (like LEFT JOIN)
MATCH (p:Patient {id: 'P001'})
OPTIONAL MATCH (p)-[r:TAKES]->(m:Medication)
RETURN p.name, collect(m.name) AS medications

-- Multiple patterns
MATCH (p:Patient)-[:DIAGNOSED_WITH]->(d:Disease)
MATCH (m:Medication)-[:TREATS]->(d)
WHERE p.id = 'P001'
RETURN DISTINCT m.name AS treatment
```

### Ordering and Pagination

```cypher
-- Order by
MATCH (p:Patient)
RETURN p
ORDER BY p.name ASC

-- Multiple sort keys
MATCH (p:Patient)
RETURN p
ORDER BY p.active DESC, p.name ASC

-- Pagination
MATCH (p:Patient)
RETURN p
ORDER BY p.name
SKIP 20
LIMIT 10
```

## 5.3 UPDATE Operations

### SET Properties

```cypher
-- Set single property
MATCH (p:Patient {id: 'P001'})
SET p.email = 'john.doe@email.com'
RETURN p

-- Set multiple properties
MATCH (p:Patient {id: 'P001'})
SET p.email = 'john@email.com',
    p.phone = '555-0123',
    p.updatedAt = datetime()
RETURN p

-- Set from map
MATCH (p:Patient {id: 'P001'})
SET p += {email: 'john@email.com', phone: '555-0123'}
RETURN p

-- Replace all properties (except identity)
MATCH (p:Patient {id: 'P001'})
SET p = {id: 'P001', name: 'John Doe Updated', active: true}
RETURN p
```

### REMOVE Properties

```cypher
-- Remove a property
MATCH (p:Patient {id: 'P001'})
REMOVE p.temporaryNote
RETURN p

-- Set to NULL (same effect)
MATCH (p:Patient {id: 'P001'})
SET p.temporaryNote = NULL
RETURN p
```

### Update Labels

```cypher
-- Add label
MATCH (p:Patient {id: 'P001'})
SET p:VIP
RETURN p

-- Remove label
MATCH (p:Patient {id: 'P001'})
REMOVE p:VIP
RETURN p
```

### Update Relationships

```cypher
-- Update relationship properties
MATCH (p:Patient {id: 'P001'})-[r:DIAGNOSED_WITH]->(d:Disease {id: 'E11'})
SET r.status = 'resolved',
    r.resolvedDate = date(),
    r.updatedAt = datetime()
RETURN r
```

## 5.4 DELETE Operations

### Delete Nodes

```cypher
-- Delete a node (will fail if has relationships)
MATCH (p:Patient {id: 'TEST001'})
DELETE p

-- Delete node and all its relationships
MATCH (p:Patient {id: 'TEST001'})
DETACH DELETE p

-- Delete multiple nodes
MATCH (p:Patient)
WHERE p.id STARTS WITH 'TEST'
DETACH DELETE p
```

### Delete Relationships

```cypher
-- Delete specific relationship
MATCH (p:Patient {id: 'P001'})-[r:TAKES]->(m:Medication {id: '6809'})
DELETE r

-- Delete all relationships of a type
MATCH ()-[r:TEMPORARY_REL]->()
DELETE r
```

### Delete All (Use with Caution!)

```cypher
-- Delete everything in the database
MATCH (n)
DETACH DELETE n
```

## 5.5 Python CRUD Implementation

Create `src/04_basic_queries.py`:

```python
"""
04_basic_queries.py
===================
Complete CRUD Operations in Neo4j

This module provides a comprehensive CypherCRUD class that handles
all basic database operations with proper error handling and
parameterized queries for security.
"""

import sys
import os
from typing import Dict, List, Any, Optional, Union
from datetime import date, datetime

sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from src.utils.connection import get_driver


class CypherCRUD:
    """
    Complete CRUD operations using Cypher queries.
    
    This class provides methods for:
    - Creating nodes and relationships
    - Reading with various filter patterns
    - Updating properties and labels
    - Deleting nodes and relationships
    - Counting and aggregating
    
    Example:
        crud = CypherCRUD()
        
        # Create
        node = crud.create_node("Patient", {"id": "P001", "name": "John"})
        
        # Read
        patient = crud.find_by_id("Patient", "P001")
        patients = crud.find_all("Patient", limit=50)
        
        # Update
        crud.update_properties("Patient", "P001", {"email": "john@example.com"})
        
        # Delete
        crud.delete_node("Patient", "P001")
    """
    
    def __init__(self, driver=None):
        """Initialize with Neo4j driver."""
        self.driver = driver or get_driver()
    
    # =========================================================
    # CREATE OPERATIONS
    # =========================================================
    
    def create_node(
        self,
        label: str,
        properties: Dict[str, Any]
    ) -> Dict[str, Any]:
        """
        Create a single node with given label and properties.
        
        Args:
            label: Node label (e.g., "Patient")
            properties: Dictionary of properties
            
        Returns:
            Created node properties
            
        Example:
            node = crud.create_node("Patient", {
                "id": "P001",
                "name": "John Doe",
                "active": True
            })
        """
        # Build property string from keys
        prop_keys = list(properties.keys())
        prop_str = ", ".join([f"{k}: ${k}" for k in prop_keys])
        
        query = f"""
        CREATE (n:{label} {{{prop_str}}})
        SET n.createdAt = datetime()
        RETURN n
        """
        
        with self.driver.session() as session:
            result = session.run(query, **properties)
            record = result.single()
            return dict(record["n"]) if record else None
    
    def create_node_with_merge(
        self,
        label: str,
        key_property: str,
        key_value: Any,
        additional_properties: Dict[str, Any] = None
    ) -> Dict[str, Any]:
        """
        Create node using MERGE (idempotent operation).
        
        Args:
            label: Node label
            key_property: Property to match on (should be unique)
            key_value: Value to match
            additional_properties: Extra properties to set
            
        Returns:
            Node data with was_created indicator
        """
        query = f"""
        MERGE (n:{label} {{{key_property}: $key_value}})
        ON CREATE SET n.createdAt = datetime()
        ON MATCH SET n.lastAccessedAt = datetime()
        """
        
        if additional_properties:
            set_clauses = ", ".join([f"n.{k} = ${k}" for k in additional_properties])
            query += f"SET {set_clauses}\n"
        
        query += """
        RETURN n, 
               n.createdAt = n.lastAccessedAt AS wasCreated
        """
        
        params = {"key_value": key_value}
        if additional_properties:
            params.update(additional_properties)
        
        with self.driver.session() as session:
            result = session.run(query, **params)
            record = result.single()
            if record:
                return {
                    "node": dict(record["n"]),
                    "was_created": record["wasCreated"]
                }
            return None
    
    def create_relationship(
        self,
        from_label: str,
        from_id: str,
        rel_type: str,
        to_label: str,
        to_id: str,
        properties: Dict[str, Any] = None
    ) -> Dict[str, Any]:
        """
        Create a relationship between two existing nodes.
        
        Args:
            from_label: Start node label
            from_id: Start node ID
            rel_type: Relationship type (e.g., "DIAGNOSED_WITH")
            to_label: End node label
            to_id: End node ID
            properties: Optional relationship properties
            
        Returns:
            Relationship details
        """
        prop_str = ""
        params = {"from_id": from_id, "to_id": to_id}
        
        if properties:
            prop_list = ", ".join([f"{k}: ${k}" for k in properties])
            prop_str = f" {{{prop_list}}}"
            params.update(properties)
        
        query = f"""
        MATCH (a:{from_label} {{id: $from_id}})
        MATCH (b:{to_label} {{id: $to_id}})
        CREATE (a)-[r:{rel_type}{prop_str}]->(b)
        SET r.createdAt = datetime()
        RETURN 
            a.id AS fromNode,
            type(r) AS relType,
            b.id AS toNode,
            properties(r) AS relProps
        """
        
        with self.driver.session() as session:
            result = session.run(query, **params)
            record = result.single()
            return record.data() if record else None
    
    # =========================================================
    # READ OPERATIONS
    # =========================================================
    
    def find_by_id(
        self,
        label: str,
        id_value: str
    ) -> Optional[Dict[str, Any]]:
        """
        Find a node by its ID property.
        
        Args:
            label: Node label
            id_value: ID value
            
        Returns:
            Node properties or None if not found
        """
        query = f"""
        MATCH (n:{label} {{id: $id}})
        RETURN n
        """
        
        with self.driver.session() as session:
            result = session.run(query, id=id_value)
            record = result.single()
            return dict(record["n"]) if record else None
    
    def find_all(
        self,
        label: str,
        limit: int = 100,
        skip: int = 0,
        order_by: str = None,
        order_desc: bool = False
    ) -> List[Dict[str, Any]]:
        """
        Find all nodes of a given label with pagination.
        
        Args:
            label: Node label
            limit: Maximum results to return
            skip: Number of results to skip
            order_by: Property to order by
            order_desc: Sort descending if True
            
        Returns:
            List of node dictionaries
        """
        order_clause = ""
        if order_by:
            direction = "DESC" if order_desc else "ASC"
            order_clause = f"ORDER BY n.{order_by} {direction}"
        
        query = f"""
        MATCH (n:{label})
        RETURN n
        {order_clause}
        SKIP $skip
        LIMIT $limit
        """
        
        with self.driver.session() as session:
            result = session.run(query, skip=skip, limit=limit)
            return [dict(r["n"]) for r in result]
    
    def find_with_filters(
        self,
        label: str,
        filters: Dict[str, Any],
        limit: int = 100
    ) -> List[Dict[str, Any]]:
        """
        Find nodes matching exact filter conditions.
        
        Args:
            label: Node label
            filters: Dictionary of property=value filters
            limit: Maximum results
            
        Returns:
            Matching nodes
        """
        if not filters:
            return self.find_all(label, limit=limit)
        
        where_clauses = [f"n.{k} = ${k}" for k in filters.keys()]
        where_str = " AND ".join(where_clauses)
        
        query = f"""
        MATCH (n:{label})
        WHERE {where_str}
        RETURN n
        LIMIT $limit
        """
        
        params = {"limit": limit, **filters}
        
        with self.driver.session() as session:
            result = session.run(query, **params)
            return [dict(r["n"]) for r in result]
    
    def find_with_where(
        self,
        label: str,
        where_clause: str,
        parameters: Dict[str, Any] = None,
        limit: int = 100
    ) -> List[Dict[str, Any]]:
        """
        Find nodes using a custom WHERE clause.
        
        Args:
            label: Node label
            where_clause: Custom WHERE condition (without WHERE keyword)
            parameters: Query parameters
            limit: Maximum results
            
        Returns:
            Matching nodes
            
        Example:
            patients = crud.find_with_where(
                "Patient",
                "n.age > $min_age AND n.active = true",
                {"min_age": 40}
            )
        """
        query = f"""
        MATCH (n:{label})
        WHERE {where_clause}
        RETURN n
        LIMIT $limit
        """
        
        params = {"limit": limit}
        if parameters:
            params.update(parameters)
        
        with self.driver.session() as session:
            result = session.run(query, **params)
            return [dict(r["n"]) for r in result]
    
    def find_related(
        self,
        label: str,
        node_id: str,
        rel_type: str = None,
        direction: str = "OUTGOING"
    ) -> List[Dict[str, Any]]:
        """
        Find nodes related to a given node.
        
        Args:
            label: Source node label
            node_id: Source node ID
            rel_type: Optional relationship type filter
            direction: "OUTGOING", "INCOMING", or "BOTH"
            
        Returns:
            Related nodes with relationship info
        """
        rel_pattern = f":{rel_type}" if rel_type else ""
        
        if direction == "OUTGOING":
            pattern = f"-[r{rel_pattern}]->(related)"
        elif direction == "INCOMING":
            pattern = f"<-[r{rel_pattern}]-(related)"
        else:
            pattern = f"-[r{rel_pattern}]-(related)"
        
        query = f"""
        MATCH (n:{label} {{id: $id}}){pattern}
        RETURN 
            labels(related)[0] AS label,
            related.id AS id,
            related.name AS name,
            type(r) AS relationship,
            properties(r) AS relProperties
        """
        
        with self.driver.session() as session:
            result = session.run(query, id=node_id)
            return [r.data() for r in result]
    
    # =========================================================
    # UPDATE OPERATIONS
    # =========================================================
    
    def update_properties(
        self,
        label: str,
        id_value: str,
        properties: Dict[str, Any]
    ) -> Optional[Dict[str, Any]]:
        """
        Update properties on a node.
        
        Args:
            label: Node label
            id_value: Node ID
            properties: Properties to update
            
        Returns:
            Updated node or None if not found
        """
        set_clauses = [f"n.{k} = ${k}" for k in properties.keys()]
        set_clauses.append("n.updatedAt = datetime()")
        set_str = ", ".join(set_clauses)
        
        query = f"""
        MATCH (n:{label} {{id: $id}})
        SET {set_str}
        RETURN n
        """
        
        params = {"id": id_value, **properties}
        
        with self.driver.session() as session:
            result = session.run(query, **params)
            record = result.single()
            return dict(record["n"]) if record else None
    
    def add_label(
        self,
        current_label: str,
        node_id: str,
        new_label: str
    ) -> bool:
        """Add a label to an existing node."""
        query = f"""
        MATCH (n:{current_label} {{id: $id}})
        SET n:{new_label}
        RETURN n
        """
        
        with self.driver.session() as session:
            result = session.run(query, id=node_id)
            return result.single() is not None
    
    def remove_label(
        self,
        current_label: str,
        node_id: str,
        label_to_remove: str
    ) -> bool:
        """Remove a label from a node."""
        query = f"""
        MATCH (n:{current_label} {{id: $id}})
        REMOVE n:{label_to_remove}
        RETURN n
        """
        
        with self.driver.session() as session:
            result = session.run(query, id=node_id)
            return result.single() is not None
    
    def remove_property(
        self,
        label: str,
        node_id: str,
        property_name: str
    ) -> bool:
        """Remove a property from a node."""
        query = f"""
        MATCH (n:{label} {{id: $id}})
        REMOVE n.{property_name}
        SET n.updatedAt = datetime()
        RETURN n
        """
        
        with self.driver.session() as session:
            result = session.run(query, id=node_id)
            return result.single() is not None
    
    # =========================================================
    # DELETE OPERATIONS
    # =========================================================
    
    def delete_node(
        self,
        label: str,
        id_value: str,
        detach: bool = True
    ) -> bool:
        """
        Delete a node.
        
        Args:
            label: Node label
            id_value: Node ID
            detach: If True, also delete all relationships
            
        Returns:
            True if deleted, False if not found
        """
        delete_cmd = "DETACH DELETE" if detach else "DELETE"
        
        query = f"""
        MATCH (n:{label} {{id: $id}})
        {delete_cmd} n
        RETURN count(*) AS deleted
        """
        
        with self.driver.session() as session:
            result = session.run(query, id=id_value)
            return result.single()["deleted"] > 0
    
    def delete_relationship(
        self,
        from_label: str,
        from_id: str,
        rel_type: str,
        to_label: str,
        to_id: str
    ) -> bool:
        """Delete a specific relationship."""
        query = f"""
        MATCH (a:{from_label} {{id: $from_id}})-[r:{rel_type}]->(b:{to_label} {{id: $to_id}})
        DELETE r
        RETURN count(*) AS deleted
        """
        
        with self.driver.session() as session:
            result = session.run(query, from_id=from_id, to_id=to_id)
            return result.single()["deleted"] > 0
    
    def delete_all_by_label(
        self,
        label: str
    ) -> int:
        """Delete all nodes with a specific label."""
        query = f"""
        MATCH (n:{label})
        DETACH DELETE n
        RETURN count(*) AS deleted
        """
        
        with self.driver.session() as session:
            result = session.run(query)
            return result.single()["deleted"]
    
    # =========================================================
    # COUNT AND AGGREGATE
    # =========================================================
    
    def count_nodes(
        self,
        label: str,
        filters: Dict[str, Any] = None
    ) -> int:
        """Count nodes matching criteria."""
        where_clause = ""
        params = {}
        
        if filters:
            where_clauses = [f"n.{k} = ${k}" for k in filters.keys()]
            where_clause = "WHERE " + " AND ".join(where_clauses)
            params = filters
        
        query = f"""
        MATCH (n:{label})
        {where_clause}
        RETURN count(n) AS count
        """
        
        with self.driver.session() as session:
            result = session.run(query, **params)
            return result.single()["count"]
    
    def count_relationships(
        self,
        rel_type: str = None
    ) -> int:
        """Count relationships of a given type."""
        type_filter = f":{rel_type}" if rel_type else ""
        
        query = f"""
        MATCH ()-[r{type_filter}]->()
        RETURN count(r) AS count
        """
        
        with self.driver.session() as session:
            result = session.run(query)
            return result.single()["count"]


# ============================================================
# DEMONSTRATION
# ============================================================

if __name__ == "__main__":
    print("=" * 60)
    print("04: Basic CRUD Operations")
    print("=" * 60)
    
    crud = CypherCRUD()
    
    # Clean up test data
    print("\n🧹 Cleaning up test data...")
    crud.delete_all_by_label("TestNode")
    
    # CREATE
    print("\n📌 CREATE Operations")
    print("-" * 40)
    
    # Create single node
    node1 = crud.create_node("TestNode", {
        "id": "T001",
        "name": "Test Node 1",
        "value": 100
    })
    print(f"   Created: {node1['name']}")
    
    # Create with merge
    result = crud.create_node_with_merge(
        "TestNode", "id", "T002",
        {"name": "Test Node 2", "value": 200}
    )
    print(f"   Merged: {result['node']['name']} (created: {result['was_created']})")
    
    # READ
    print("\n📌 READ Operations")
    print("-" * 40)
    
    # Find by ID
    found = crud.find_by_id("TestNode", "T001")
    print(f"   Found by ID: {found['name']}")
    
    # Find all
    all_nodes = crud.find_all("TestNode")
    print(f"   All TestNodes: {len(all_nodes)}")
    
    # Find with filters
    filtered = crud.find_with_filters("TestNode", {"value": 100})
    print(f"   Filtered (value=100): {len(filtered)}")
    
    # UPDATE
    print("\n📌 UPDATE Operations")
    print("-" * 40)
    
    # Update properties
    updated = crud.update_properties("TestNode", "T001", {"value": 150})
    print(f"   Updated value: {updated['value']}")
    
    # Add label
    crud.add_label("TestNode", "T001", "Special")
    print("   Added label: Special")
    
    # DELETE
    print("\n📌 DELETE Operations")
    print("-" * 40)
    
    # Delete node
    deleted = crud.delete_node("TestNode", "T001")
    print(f"   Deleted T001: {deleted}")
    
    # Count remaining
    count = crud.count_nodes("TestNode")
    print(f"   Remaining TestNodes: {count}")
    
    # Cleanup
    crud.delete_all_by_label("TestNode")
    
    print("\n" + "=" * 60)
    print("✅ CRUD operations completed successfully!")
    print("=" * 60)
```

---

# 6. Advanced Cypher Queries

## 6.1 Variable-Length Paths

Variable-length paths allow you to match patterns of unknown depth.

```cypher
-- Syntax: [*min..max]
-- [*] = any length
-- [*2] = exactly 2 hops
-- [*1..3] = 1 to 3 hops
-- [*..5] = up to 5 hops

-- Find all connected nodes within 3 hops
MATCH path = (p:Patient {id: 'P001'})-[*1..3]-(connected)
RETURN connected, length(path) AS distance
ORDER BY distance

-- With specific relationship types
MATCH path = (p:Patient {id: 'P001'})-[:DIAGNOSED_WITH|TAKES*1..2]-(end)
RETURN DISTINCT labels(end)[0] AS type, end.name AS name

-- Collect path information
MATCH path = (p:Patient {id: 'P001'})-[*1..3]-(connected)
WHERE p <> connected
RETURN 
    connected.name AS node,
    length(path) AS distance,
    [node IN nodes(path) | node.name] AS pathNodes,
    [rel IN relationships(path) | type(rel)] AS pathRelationships
ORDER BY distance
LIMIT 20
```

## 6.2 Shortest Path Algorithms

```cypher
-- Find the shortest path between two nodes
MATCH path = shortestPath(
    (p1:Patient {id: 'P001'})-[*]-(p2:Patient {id: 'P003'})
)
RETURN path, length(path) AS distance

-- All shortest paths (if multiple exist with same length)
MATCH paths = allShortestPaths(
    (p1:Patient {id: 'P001'})-[*]-(p2:Patient {id: 'P003'})
)
RETURN paths

-- Shortest path with relationship type filter
MATCH path = shortestPath(
    (p:Patient {id: 'P001'})-[:DIAGNOSED_WITH|TREATS*]-(m:Medication)
)
RETURN 
    [node IN nodes(path) | node.name] AS pathNodes,
    length(path) AS distance

-- Weighted shortest path (with GDS)
-- See Section 14 for Dijkstra's algorithm
```

## 6.3 Aggregation Functions

```cypher
-- Count
MATCH (p:Patient)
RETURN count(p) AS totalPatients

-- Count distinct
MATCH (p:Patient)-[:DIAGNOSED_WITH]->(d:Disease)
RETURN count(DISTINCT d) AS uniqueDiseases

-- Collect (creates a list)
MATCH (p:Patient {id: 'P001'})-[:TAKES]->(m:Medication)
RETURN collect(m.name) AS medications

-- Collect with limit
MATCH (p:Patient)-[:DIAGNOSED_WITH]->(d:Disease)
RETURN d.name, collect(p.name)[0..5] AS samplePatients

-- Statistical functions
MATCH (p:Patient)
WHERE p.age IS NOT NULL
RETURN 
    avg(p.age) AS avgAge,
    min(p.age) AS minAge,
    max(p.age) AS maxAge,
    stDev(p.age) AS stdDevAge,
    percentileCont(p.age, 0.5) AS medianAge

-- Group by
MATCH (p:Patient)-[:DIAGNOSED_WITH]->(d:Disease)
RETURN 
    d.category AS category,
    count(DISTINCT p) AS patientCount,
    collect(DISTINCT d.name) AS diseases
ORDER BY patientCount DESC
```

## 6.4 Subqueries

```cypher
-- CALL subquery (correlated)
MATCH (p:Patient)
CALL {
    WITH p
    MATCH (p)-[:DIAGNOSED_WITH]->(d:Disease)
    RETURN count(d) AS diseaseCount
}
RETURN p.name, diseaseCount
ORDER BY diseaseCount DESC

-- EXISTS subquery for filtering
MATCH (p:Patient)
WHERE EXISTS {
    MATCH (p)-[:DIAGNOSED_WITH]->(d:Disease {severity: 'severe'})
}
RETURN p.name AS patientsWithSevereDiagnosis

-- COUNT subquery
MATCH (d:Disease)
WHERE COUNT {
    MATCH (:Patient)-[:DIAGNOSED_WITH]->(d)
} > 10
RETURN d.name AS commonDiseases

-- COLLECT subquery
MATCH (p:Patient)
RETURN p.name, COLLECT {
    MATCH (p)-[:TAKES]->(m:Medication)
    RETURN m.name
} AS medications
```

## 6.5 UNION and UNWIND

```cypher
-- UNION (combines results, removes duplicates)
MATCH (p:Patient {id: 'P001'})-[:DIAGNOSED_WITH]->(d:Disease)
RETURN 'Disease' AS type, d.name AS name

UNION

MATCH (p:Patient {id: 'P001'})-[:TAKES]->(m:Medication)
RETURN 'Medication' AS type, m.name AS name

-- UNION ALL (keeps duplicates, faster)
MATCH (p:Patient {id: 'P001'})-[:DIAGNOSED_WITH]->(d:Disease)
RETURN 'Disease' AS type, d.name AS name

UNION ALL

MATCH (p:Patient {id: 'P001'})-[:TAKES]->(m:Medication)
RETURN 'Medication' AS type, m.name AS name

-- UNWIND (expand list to rows)
WITH ['Diabetes', 'Hypertension', 'Asthma'] AS conditions
UNWIND conditions AS condition
MATCH (d:Disease)
WHERE d.name CONTAINS condition
RETURN d

-- UNWIND for batch operations
UNWIND $patients AS patientData
CREATE (p:Patient {
    id: patientData.id,
    name: patientData.name
})
RETURN count(*) AS created
```

## 6.6 Pattern Comprehensions

```cypher
-- Pattern comprehension syntax: [(pattern) | expression]
MATCH (p:Patient {id: 'P001'})
RETURN 
    p.name AS patient,
    [(p)-[:DIAGNOSED_WITH]->(d) | d.name] AS diseases,
    [(p)-[:TAKES]->(m) | {
        name: m.name,
        class: m.drugClass
    }] AS medications

-- With WHERE filter
MATCH (p:Patient {id: 'P001'})
RETURN 
    p.name AS patient,
    [(p)-[r:TAKES]->(m) WHERE r.status = 'active' | m.name] AS activeMeds
```

## 6.7 CASE Expressions

```cypher
-- Simple CASE
MATCH (p:Patient)
RETURN 
    p.name,
    CASE p.gender
        WHEN 'Male' THEN 'M'
        WHEN 'Female' THEN 'F'
        ELSE 'O'
    END AS genderCode

-- Generic CASE
MATCH (p:Patient)
RETURN 
    p.name,
    CASE 
        WHEN p.age < 18 THEN 'Minor'
        WHEN p.age < 40 THEN 'Young Adult'
        WHEN p.age < 65 THEN 'Middle Age'
        ELSE 'Senior'
    END AS ageGroup,
    CASE
        WHEN p.active AND size((p)-[:DIAGNOSED_WITH]->()) > 0 THEN 'Active with conditions'
        WHEN p.active THEN 'Active healthy'
        ELSE 'Inactive'
    END AS status
```

## 6.8 List Functions

```cypher
-- List operations
MATCH (p:Patient {id: 'P001'})-[:DIAGNOSED_WITH]->(d:Disease)
WITH collect(d.name) AS diseases
RETURN 
    diseases,
    size(diseases) AS count,
    head(diseases) AS first,
    last(diseases) AS last,
    tail(diseases) AS rest

-- List filtering
MATCH (p:Patient {id: 'P001'})-[:DIAGNOSED_WITH]->(d:Disease)
WITH collect(d) AS diseases
RETURN [d IN diseases WHERE d.severity = 'severe' | d.name] AS severeDiseases

-- List transformation
MATCH (p:Patient {id: 'P001'})-[:TAKES]->(m:Medication)
WITH collect(m.name) AS meds
RETURN [med IN meds | toUpper(med)] AS upperCaseMeds

-- Reduce
MATCH (p:Patient {id: 'P001'})-[:TAKES]->(m:Medication)
WITH collect(m.name) AS meds
RETURN reduce(s = '', med IN meds | s + med + ', ') AS medString
```

Patient)
        WHERE p.cluster IS NOT NULL
        WITH p.cluster AS clusterId, 
             count(p) AS patientCount,
             avg(p.age) AS avgAge,
             collect(p.id)[0..5] AS samplePatients
        
        // Create cluster summary nodes
        CREATE (cs:ClusterSummary {
            id: 'cluster_' + toString(clusterId),
            clusterId: clusterId,
            patientCount: patientCount,
            avgAge: avgAge,
            samplePatients: samplePatients,
            createdAt: datetime()
        })
        
        // Link patients to their cluster
        WITH cs, clusterId
        MATCH (p:Patient {cluster: clusterId})
        CREATE (p)-[:BELONGS_TO]->(cs)
        
        RETURN count(DISTINCT cs) AS summariesCreated
        """
        
        with self.driver.session() as session:
            result = session.run(query)
            return result.single()["summariesCreated"]
    
    def query_at_cluster_level(self, disease_id: str):
        """
        Query cluster summaries instead of individual patients.
        Much faster for large graphs.
        """
        query = """
        // First, get the disease
        MATCH (d:Disease {id: $diseaseId})
        
        // Get cluster summaries for patients with this disease
        MATCH (p:Patient)-[:DIAGNOSED_WITH]->(d)
        MATCH (p)-[:BELONGS_TO]->(cs:ClusterSummary)
        
        WITH cs, count(p) AS patientsWithDisease
        RETURN 
            cs.clusterId AS cluster,
            cs.patientCount AS totalPatients,
            patientsWithDisease,
            cs.avgAge AS avgAge
        ORDER BY patientsWithDisease DESC
        LIMIT 10
        """
        
        with self.driver.session() as session:
            result = session.run(query, diseaseId=disease_id)
            return [r.data() for r in result]
    
    def get_federated_embeddings(self, entity_id: str, max_nodes: int = 1000):
        """
        Get embeddings in a federated way for large entity clusters.
        
        Instead of loading all 10K+ connected nodes, we:
        1. Sample representative nodes
        2. Get cluster centroids
        3. Combine for efficient representation
        """
        query = """
        MATCH (e {id: $entityId})<-[r]-(connected)
        WITH connected, rand() AS random
        ORDER BY random
        LIMIT $maxNodes
        
        // Get embeddings where available
        RETURN 
            connected.id AS id,
            connected.name AS name,
            connected.embedding AS embedding,
            labels(connected)[0] AS type
        """
        
        with self.driver.session() as session:
            result = session.run(query, entityId=entity_id, maxNodes=max_nodes)
            return [r.data() for r in result]


# ============================================================
# DEMONSTRATION
# ============================================================

if __name__ == "__main__":
    print("=" * 60)
    print("26: Large-Scale Graph Clustering")
    print("=" * 60)
    
    print("\n📌 Large-Scale Strategies")
    print("-" * 40)
    print("""
    When dealing with 10,000+ nodes around a single entity:
    
    1. HIERARCHICAL CLUSTERING
       - Group related nodes into clusters
       - Create summary nodes for each cluster
       - Query summaries for fast overview
       - Drill down only when needed
    
    2. SAMPLING STRATEGIES
       - Random sampling for representative subsets
       - Stratified sampling by node properties
       - Importance-based sampling (PageRank)
    
    3. FEDERATED EMBEDDINGS
       - Don't load all embeddings at once
       - Use cluster centroids
       - Lazy loading of detailed embeddings
    
    4. MATERIALIZED VIEWS
       - Pre-compute common aggregations
       - Store as summary nodes
       - Update incrementally
    """)
    
    print("\n" + "=" * 60)
    print("✅ Large-scale clustering concepts explained!")
    print("=" * 60)
```

---

# 27. Neo4j Internal Architecture

## 27.1 Storage Architecture

Understanding how Neo4j stores data helps optimize your queries.

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                       NEO4J STORAGE ARCHITECTURE                            │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  NATIVE GRAPH STORAGE                                                       │
│  ════════════════════                                                       │
│                                                                             │
│  1. NODE STORE (neostore.nodestore.db)                                     │
│     ┌─────────────────────────────────────────────────────────────────┐    │
│     │ Node Record (15 bytes fixed size)                               │    │
│     │ ┌────────┬───────────────┬───────────────┬─────────────────────┐│    │
│     │ │ InUse  │ First Rel ID  │ First Prop ID │ Label Pointer       ││    │
│     │ │ 1 byte │ 5 bytes       │ 5 bytes       │ 4 bytes             ││    │
│     │ └────────┴───────────────┴───────────────┴─────────────────────┘│    │
│     └─────────────────────────────────────────────────────────────────┘    │
│                                                                             │
│     • Fixed-size records enable O(1) lookup by ID                          │
│     • First Rel ID points to first relationship                            │
│     • First Prop ID points to first property                               │
│     • Label Pointer references label store                                  │
│                                                                             │
│  2. RELATIONSHIP STORE (neostore.relationshipstore.db)                     │
│     ┌─────────────────────────────────────────────────────────────────┐    │
│     │ Relationship Record (34 bytes fixed size)                       │    │
│     │ ┌────────┬────────┬────────┬──────────┬──────────┬────────────┐│    │
│     │ │ InUse  │ Start  │ End    │ Rel Type │ Start    │ End        ││    │
│     │ │ 1 byte │ Node   │ Node   │          │ Prev/Next│ Prev/Next  ││    │
│     │ │        │ 5 bytes│ 5 bytes│ 4 bytes  │ 5+5 bytes│ 5+5 bytes  ││    │
│     │ └────────┴────────┴────────┴──────────┴──────────┴────────────┘│    │
│     └─────────────────────────────────────────────────────────────────┘    │
│                                                                             │
│     • Doubly-linked list per node                                          │
│     • Can traverse in either direction                                     │
│     • Start/End prev/next enable efficient relationship iteration          │
│                                                                             │
│  3. PROPERTY STORE (neostore.propertystore.db)                             │
│     ┌─────────────────────────────────────────────────────────────────┐    │
│     │ Property Record (41 bytes)                                      │    │
│     │ ┌────────┬────────────┬─────────────────┬───────────────────┐  │    │
│     │ │ InUse  │ Prop Block │ Prop Block      │ Next Prop ID      │  │    │
│     │ │ 1 byte │ (4 props)  │ (4 props)       │ 5 bytes           │  │    │
│     │ └────────┴────────────┴─────────────────┴───────────────────┘  │    │
│     └─────────────────────────────────────────────────────────────────┘    │
│                                                                             │
│     • Each record holds up to 4 property blocks                            │
│     • Properties form a linked list                                        │
│     • Large values stored in dynamic string/array stores                   │
│                                                                             │
│  4. DYNAMIC STORES                                                          │
│     • String Store: For strings > 15 bytes                                 │
│     • Array Store: For arrays                                              │
│     • Label Token Store: For label definitions                             │
│     • Rel Type Token Store: For relationship type definitions              │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘
```

## 27.2 Index-Free Adjacency

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                       INDEX-FREE ADJACENCY                                  │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  WHY GRAPH TRAVERSAL IS FAST                                               │
│  ════════════════════════════                                               │
│                                                                             │
│  RELATIONAL DATABASE (With JOINs):                                         │
│  ─────────────────────────────────                                         │
│                                                                             │
│  Query: Find friends of friends                                             │
│                                                                             │
│  Step 1: Find person → O(log n) index lookup                               │
│  Step 2: Scan FRIENDS table for matches → O(m) where m = all friendships   │
│  Step 3: For each friend, scan again → O(m) per friend                     │
│  Total: O(log n + m × friends)  -- Gets slower as data grows!              │
│                                                                             │
│  NEO4J (Index-Free Adjacency):                                             │
│  ─────────────────────────────                                              │
│                                                                             │
│  Query: Find friends of friends                                             │
│                                                                             │
│  Step 1: Find person → O(1) node ID lookup                                 │
│  Step 2: Follow relationship pointer → O(k) where k = this person's friends│
│  Step 3: For each friend, follow pointer → O(k) per friend                 │
│  Total: O(1 + k × avg_friends)  -- Constant time per hop!                  │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │                                                                      │   │
│  │   Node A ──┬── first_rel ──► Rel 1 ◄──► Rel 2 ◄──► Rel 3           │   │
│  │            │                   │          │          │              │   │
│  │            │                   ▼          ▼          ▼              │   │
│  │            │               Node B      Node C     Node D            │   │
│  │            │                                                        │   │
│  │            │   Relationships are POINTERS, not lookups!            │   │
│  │            │                                                        │   │
│  └────────────┴────────────────────────────────────────────────────────┘   │
│                                                                             │
│  KEY INSIGHT:                                                               │
│  • Each node stores a pointer to its first relationship                    │
│  • Each relationship stores pointers to next relationships                 │
│  • Traversing = following pointers in memory                               │
│  • Performance depends on LOCAL data, not TOTAL data                       │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘
```

## 27.3 Page Cache and Memory

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                       MEMORY ARCHITECTURE                                   │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │                         JVM HEAP                                     │   │
│  │                                                                      │   │
│  │   • Transaction state                                                │   │
│  │   • Query execution                                                  │   │
│  │   • Result aggregation                                               │   │
│  │   • Object cache                                                     │   │
│  │                                                                      │   │
│  │   Config: dbms.memory.heap.initial_size / max_size                  │   │
│  │   Recommendation: 1GB - 4GB typically sufficient                     │   │
│  │                                                                      │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │                       PAGE CACHE                                     │   │
│  │                                                                      │   │
│  │   • Cached store files (nodes, relationships, properties)           │   │
│  │   • LRU eviction when full                                          │   │
│  │   • Larger = better performance                                      │   │
│  │                                                                      │   │
│  │   Config: dbms.memory.pagecache.size                                │   │
│  │                                                                      │   │
│  │   SIZING FORMULA:                                                    │   │
│  │   page_cache >= total_store_size × 1.2                              │   │
│  │                                                                      │   │
│  │   For 5 milli-second queries:                                        │   │
│  │   • Page cache should fit entire working set                        │   │
│  │   • Hot data should never need disk reads                           │   │
│  │   • Monitor page faults and increase if needed                      │   │
│  │                                                                      │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │                       DISK (Store Files)                             │   │
│  │                                                                      │   │
│  │   neostore.nodestore.db                                             │   │
│  │   neostore.relationshipstore.db                                     │   │
│  │   neostore.propertystore.db                                         │   │
│  │   neostore.propertystore.db.strings                                 │   │
│  │   neostore.propertystore.db.arrays                                  │   │
│  │   schema/ (indexes, constraints)                                    │   │
│  │   transaction logs                                                   │   │
│  │                                                                      │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘
```

## 27.4 How 5ms Query Performance is Achieved

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                       5 MILLISECOND QUERY PATH                              │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  QUERY: MATCH (p:Patient {id: 'P001'})-[:DIAGNOSED_WITH]->(d) RETURN d     │
│                                                                             │
│  STEP 1: INDEX LOOKUP (< 0.5 ms)                                           │
│  ═══════════════════════════════                                            │
│  • Query hits B-tree index on Patient.id                                   │
│  • B-tree lookup: O(log n) comparisons                                     │
│  • For 1M patients: ~20 comparisons                                        │
│  • Each comparison: ~25 microseconds (cached)                              │
│  • Result: Node ID for P001                                                │
│                                                                             │
│  STEP 2: NODE FETCH (< 0.1 ms)                                             │
│  ══════════════════════════════                                             │
│  • Calculate offset: node_id × 15 bytes                                    │
│  • Direct memory read (page cache)                                         │
│  • Get first_relationship pointer                                          │
│                                                                             │
│  STEP 3: RELATIONSHIP TRAVERSAL (< 1 ms)                                   │
│  ═══════════════════════════════════════                                    │
│  • Follow first_rel pointer to relationship store                          │
│  • Scan relationship chain                                                  │
│  • For each DIAGNOSED_WITH: get end_node                                   │
│  • Typically < 100 relationships to scan                                   │
│                                                                             │
│  STEP 4: TARGET NODE FETCH (< 0.5 ms)                                      │
│  ════════════════════════════════════                                       │
│  • For each matched relationship                                           │
│  • Fetch end node (Disease)                                                │
│  • Load properties                                                         │
│                                                                             │
│  STEP 5: RESULT ASSEMBLY (< 1 ms)                                          │
│  ════════════════════════════════                                           │
│  • Convert to result format                                                │
│  • Send to client                                                          │
│                                                                             │
│  TOTAL: ~2-5 ms (when page cache is warm)                                  │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │  WHAT MAKES IT FAST:                                                 │   │
│  │                                                                      │   │
│  │  ✓ Index for initial lookup                                         │   │
│  │  ✓ Fixed-size records = direct offset calculation                   │   │
│  │  ✓ Pointer-based traversal (no joins)                               │   │
│  │  ✓ Page cache keeps hot data in memory                              │   │
│  │  ✓ Query only touches LOCAL data                                    │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘
```

---

# 28. Performance Optimization

## 28.1 Query Optimization

```cypher
-- Use EXPLAIN to see query plan without executing
EXPLAIN
MATCH (p:Patient {id: 'P001'})-[:DIAGNOSED_WITH]->(d:Disease)
RETURN p, d

-- Use PROFILE to see actual execution statistics
PROFILE
MATCH (p:Patient {id: 'P001'})-[:DIAGNOSED_WITH]->(d:Disease)
RETURN p, d

-- Results show:
-- • DbHits: Database operations performed
-- • Rows: Number of records at each step
-- • Time: Execution time per step
-- • Memory: Memory used
```

## 28.2 Common Optimization Patterns

```cypher
-- BAD: Full label scan
MATCH (p:Patient)
WHERE p.id = 'P001'
RETURN p

-- GOOD: Index-backed lookup (create index first)
-- CREATE INDEX patient_id FOR (p:Patient) ON (p.id)
MATCH (p:Patient {id: 'P001'})
RETURN p

-- BAD: Cartesian product
MATCH (a:Patient), (b:Disease)
WHERE EXISTS { MATCH (a)-[:DIAGNOSED_WITH]->(b) }
RETURN a, b

-- GOOD: Direct pattern match
MATCH (a:Patient)-[:DIAGNOSED_WITH]->(b:Disease)
RETURN a, b

-- BAD: Collecting all then filtering
MATCH (p:Patient)-[:DIAGNOSED_WITH]->(d:Disease)
WITH p, collect(d) AS diseases
WHERE size(diseases) > 3
RETURN p

-- GOOD: Early filtering
MATCH (p:Patient)
WHERE size((p)-[:DIAGNOSED_WITH]->()) > 3
MATCH (p)-[:DIAGNOSED_WITH]->(d:Disease)
RETURN p, collect(d) AS diseases

-- Optimize aggregations with early limiting
MATCH (p:Patient)-[r:DIAGNOSED_WITH]->(d:Disease)
WITH p, count(d) AS diseaseCount
WHERE diseaseCount > 5
WITH p ORDER BY diseaseCount DESC LIMIT 100
MATCH (p)-[:DIAGNOSED_WITH]->(d:Disease)
RETURN p.name, collect(d.name) AS diseases
```

## 28.3 Performance Monitoring Queries

```cypher
-- Check database statistics
CALL db.stats.retrieve('TOKENS') YIELD data
RETURN data

-- List running queries
CALL dbms.listQueries()
YIELD query, elapsedTimeMillis, status
RETURN query, elapsedTimeMillis, status
ORDER BY elapsedTimeMillis DESC

-- Check index utilization
CALL db.indexes() YIELD name, type, labelsOrTypes, properties
RETURN name, type, labelsOrTypes, properties

-- Memory usage
CALL dbms.listPools()
YIELD pool, heapMemoryUsed, heapMemoryMax
RETURN pool, heapMemoryUsed, heapMemoryMax
```

---

# 29. Enterprise Deployment

## 29.1 Cluster Architecture

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                       NEO4J CLUSTER (Enterprise)                            │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│                           ┌─────────────────┐                               │
│                           │  Load Balancer  │                               │
│                           └────────┬────────┘                               │
│                                    │                                        │
│              ┌─────────────────────┼─────────────────────┐                  │
│              │                     │                     │                  │
│              ▼                     ▼                     ▼                  │
│    ┌─────────────────┐   ┌─────────────────┐   ┌─────────────────┐         │
│    │    PRIMARY      │   │    SECONDARY    │   │    SECONDARY    │         │
│    │   (Leader)      │   │   (Follower)    │   │   (Follower)    │         │
│    │                 │   │                 │   │                 │         │
│    │ • All writes    │   │ • Reads only    │   │ • Reads only    │         │
│    │ • Reads         │   │ • Standby for   │   │ • Standby for   │         │
│    │ • Coordinates   │   │   failover      │   │   failover      │         │
│    │   replication   │   │ • Async sync    │   │ • Async sync    │         │
│    └─────────────────┘   └─────────────────┘   └─────────────────┘         │
│                                                                             │
│    CAUSAL CONSISTENCY:                                                      │
│    • Write to primary                                                       │
│    • Get bookmark from write transaction                                    │
│    • Pass bookmark to read query                                           │
│    • Read waits until secondary has applied that bookmark                  │
│    • Guarantees "read your own writes"                                     │
│                                                                             │
│    AUTOMATIC SCALING:                                                       │
│    • Add read replicas for read scaling                                    │
│    • Neo4j automatically routes reads                                      │
│    • Writes always go to primary                                           │
│    • Failover within seconds if primary fails                              │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘
```

## 29.2 Configuration for Production

```yaml
# neo4j.conf for production
# ===========================

# Memory settings
dbms.memory.heap.initial_size=4G
dbms.memory.heap.max_size=4G
dbms.memory.pagecache.size=8G

# JVM settings
dbms.jvm.additional=-XX:+UseG1GC
dbms.jvm.additional=-XX:+ParallelRefProcEnabled
dbms.jvm.additional=-XX:MaxGCPauseMillis=200

# Transaction log
dbms.tx_log.rotation.retention_policy=2 days
dbms.checkpoint.interval.time=15m
dbms.checkpoint.interval.tx=100000

# Security
dbms.security.auth_enabled=true
dbms.ssl.policy.bolt.enabled=true
dbms.ssl.policy.https.enabled=true

# Network
dbms.connector.bolt.listen_address=:7687
dbms.connector.http.listen_address=:7474
dbms.connector.bolt.thread_pool_max_size=400

# Cluster (Enterprise)
causal_clustering.minimum_core_cluster_size_at_formation=3
causal_clustering.minimum_core_cluster_size_at_runtime=3
causal_clustering.initial_discovery_members=core01:5000,core02:5000,core03:5000
```

---

# 30. REST API Development

## 30.1 FastAPI Server Implementation

Create `src/30_api_server.py`:

```python
"""
30_api_server.py
================
FastAPI REST API for Neo4j Knowledge Graph

Endpoints:
- POST /nodes - Create node
- GET /nodes/{label}/{id} - Get node
- PUT /nodes/{label}/{id} - Update node
- DELETE /nodes/{label}/{id} - Delete node
- POST /relationships - Create relationship
- POST /query - Execute Cypher query
- POST /rag - Graph RAG query
- GET /health - Health check
- GET /stats - Database statistics

Run: uvicorn src.30_api_server:app --reload --port 8000
"""

import os
import sys
from typing import Dict, List, Any, Optional
from datetime import datetime

from fastapi import FastAPI, HTTPException, Query, Depends
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel, Field

sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from src.utils.connection import Neo4jConnection


# ============================================================
# PYDANTIC MODELS
# ============================================================

class NodeCreate(BaseModel):
    """Request body for node creation."""
    label: str = Field(..., example="Patient")
    properties: Dict[str, Any] = Field(..., example={"id": "P001", "name": "John Doe"})


class NodeUpdate(BaseModel):
    """Request body for node update."""
    properties: Dict[str, Any] = Field(..., example={"email": "john@example.com"})


class RelationshipCreate(BaseModel):
    """Request body for relationship creation."""
    from_label: str = Field(..., example="Patient")
    from_id: str = Field(..., example="P001")
    rel_type: str = Field(..., example="DIAGNOSED_WITH")
    to_label: str = Field(..., example="Disease")
    to_id: str = Field(..., example="E11")
    properties: Optional[Dict[str, Any]] = Field(None, example={"date": "2024-01-15"})


class CypherQuery(BaseModel):
    """Request body for Cypher query execution."""
    query: str = Field(..., example="MATCH (n:Patient) RETURN n LIMIT 10")
    parameters: Optional[Dict[str, Any]] = Field(None, example={"limit": 10})


class RAGQuery(BaseModel):
    """Request body for RAG query."""
    question: str = Field(..., example="What medications treat diabetes?")
    top_k: int = Field(5, ge=1, le=20, example=5)


class HealthResponse(BaseModel):
    """Health check response."""
    status: str
    version: str
    node_count: int
    relationship_count: int
    timestamp: str


class StatsResponse(BaseModel):
    """Database statistics response."""
    labels: List[str]
    relationship_types: List[str]
    node_counts: Dict[str, int]
    index_count: int
    constraint_count: int


# ============================================================
# DATABASE CONNECTION
# ============================================================

_connection: Optional[Neo4jConnection] = None


def get_db() -> Neo4jConnection:
    """Dependency to get database connection."""
    global _connection
    if _connection is None:
        _connection = Neo4jConnection()
        _connection.connect()
    return _connection


# ============================================================
# FASTAPI APPLICATION
# ============================================================

app = FastAPI(
    title="Neo4j Knowledge Graph API",
    description="REST API for Neo4j graph database operations",
    version="1.0.0",
    docs_url="/docs",
    redoc_url="/redoc"
)

# CORS middleware
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)


# ============================================================
# NODE ENDPOINTS
# ============================================================

@app.post("/nodes", tags=["Nodes"])
async def create_node(
    node: NodeCreate,
    db: Neo4jConnection = Depends(get_db)
) -> Dict[str, Any]:
    """
    Create a new node.
    
    - **label**: Node label (e.g., Patient, Disease)
    - **properties**: Dictionary of properties (must include 'id')
    """
    if "id" not in node.properties:
        raise HTTPException(400, "Properties must include 'id'")
    
    prop_keys = list(node.properties.keys())
    prop_str = ", ".join([f"{k}: ${k}" for k in prop_keys])
    
    query = f"""
    CREATE (n:{node.label} {{{prop_str}}})
    SET n.createdAt = datetime()
    RETURN n
    """
    
    try:
        results = db.execute_query(query, node.properties)
        if results:
            return {"status": "created", "node": results[0]["n"]}
        raise HTTPException(500, "Failed to create node")
    except Exception as e:
        raise HTTPException(500, str(e))


@app.get("/nodes/{label}/{node_id}", tags=["Nodes"])
async def get_node(
    label: str,
    node_id: str,
    db: Neo4jConnection = Depends(get_db)
) -> Dict[str, Any]:
    """Get a node by label and ID."""
    query = f"""
    MATCH (n:{label} {{id: $id}})
    RETURN n
    """
    
    results = db.execute_query(query, {"id": node_id})
    if not results:
        raise HTTPException(404, f"Node not found: {label}/{node_id}")
    
    return {"node": results[0]["n"]}


@app.put("/nodes/{label}/{node_id}", tags=["Nodes"])
async def update_node(
    label: str,
    node_id: str,
    update: NodeUpdate,
    db: Neo4jConnection = Depends(get_db)
) -> Dict[str, Any]:
    """Update node properties."""
    set_clauses = [f"n.{k} = ${k}" for k in update.properties.keys()]
    set_clauses.append("n.updatedAt = datetime()")
    set_str = ", ".join(set_clauses)
    
    query = f"""
    MATCH (n:{label} {{id: $id}})
    SET {set_str}
    RETURN n
    """
    
    params = {"id": node_id, **update.properties}
    results = db.execute_query(query, params)
    
    if not results:
        raise HTTPException(404, f"Node not found: {label}/{node_id}")
    
    return {"status": "updated", "node": results[0]["n"]}


@app.delete("/nodes/{label}/{node_id}", tags=["Nodes"])
async def delete_node(
    label: str,
    node_id: str,
    detach: bool = Query(True, description="Also delete relationships"),
    db: Neo4jConnection = Depends(get_db)
) -> Dict[str, Any]:
    """Delete a node."""
    delete_cmd = "DETACH DELETE" if detach else "DELETE"
    
    query = f"""
    MATCH (n:{label} {{id: $id}})
    {delete_cmd} n
    RETURN count(*) AS deleted
    """
    
    results = db.execute_query(query, {"id": node_id})
    deleted = results[0]["deleted"] if results else 0
    
    if deleted == 0:
        raise HTTPException(404, f"Node not found: {label}/{node_id}")
    
    return {"status": "deleted", "count": deleted}


# ============================================================
# RELATIONSHIP ENDPOINTS
# ============================================================

@app.post("/relationships", tags=["Relationships"])
async def create_relationship(
    rel: RelationshipCreate,
    db: Neo4jConnection = Depends(get_db)
) -> Dict[str, Any]:
    """Create a relationship between two nodes."""
    prop_str = ""
    params = {"from_id": rel.from_id, "to_id": rel.to_id}
    
    if rel.properties:
        prop_list = ", ".join([f"{k}: ${k}" for k in rel.properties])
        prop_str = f" {{{prop_list}}}"
        params.update(rel.properties)
    
    query = f"""
    MATCH (a:{rel.from_label} {{id: $from_id}})
    MATCH (b:{rel.to_label} {{id: $to_id}})
    CREATE (a)-[r:{rel.rel_type}{prop_str}]->(b)
    SET r.createdAt = datetime()
    RETURN a.id AS from_node, type(r) AS rel_type, b.id AS to_node
    """
    
    results = db.execute_query(query, params)
    if not results:
        raise HTTPException(404, "One or both nodes not found")
    
    return {"status": "created", "relationship": results[0]}


# ============================================================
# QUERY ENDPOINTS
# ============================================================

@app.post("/query", tags=["Query"])
async def execute_query(
    cypher: CypherQuery,
    db: Neo4jConnection = Depends(get_db)
) -> Dict[str, Any]:
    """
    Execute a Cypher query.
    
    **Warning**: This endpoint executes arbitrary Cypher. 
    In production, implement proper authorization and query validation.
    """
    # Basic validation - block dangerous operations
    query_upper = cypher.query.upper()
    if any(cmd in query_upper for cmd in ["DROP", "CREATE DATABASE", "STOP DATABASE"]):
        raise HTTPException(403, "Administrative commands not allowed")
    
    try:
        results = db.execute_query(
            cypher.query,
            cypher.parameters or {}
        )
        return {
            "status": "success",
            "count": len(results),
            "results": results
        }
    except Exception as e:
        raise HTTPException(400, f"Query error: {str(e)}")


# ============================================================
# RAG ENDPOINT
# ============================================================

@app.post("/rag", tags=["RAG"])
async def rag_query(
    rag: RAGQuery,
    db: Neo4jConnection = Depends(get_db)
) -> Dict[str, Any]:
    """
    Execute a Graph RAG query.
    
    1. Embeds the question
    2. Finds similar chunks
    3. Expands graph context
    4. Returns assembled context
    """
    try:
        from src.23_graph_rag_basics import GraphRAG
        
        graph_rag = GraphRAG(driver=db.driver)
        result = graph_rag.query(rag.question, top_k_chunks=rag.top_k)
        
        return {
            "question": result.question,
            "context": result.context,
            "chunks_found": len(result.chunks),
            "entities_found": len(result.entities),
            "relationships_found": len(result.relationships)
        }
    except ImportError:
        # Fallback without RAG
        return {
            "error": "RAG module not available",
            "question": rag.question
        }
    except Exception as e:
        raise HTTPException(500, f"RAG error: {str(e)}")


# ============================================================
# UTILITY ENDPOINTS
# ============================================================

@app.get("/health", response_model=HealthResponse, tags=["Utility"])
async def health_check(
    db: Neo4jConnection = Depends(get_db)
) -> HealthResponse:
    """Check database health and get basic stats."""
    health = db.health_check()
    
    return HealthResponse(
        status=health["status"],
        version=health.get("version", "unknown"),
        node_count=health.get("node_count", 0),
        relationship_count=health.get("relationship_count", 0),
        timestamp=datetime.utcnow().isoformat()
    )


@app.get("/stats", response_model=StatsResponse, tags=["Utility"])
async def get_stats(
    db: Neo4jConnection = Depends(get_db)
) -> StatsResponse:
    """Get detailed database statistics."""
    schema = db.get_schema()
    
    # Get node counts per label
    node_counts = {}
    for label in schema["labels"]:
        results = db.execute_query(
            f"MATCH (n:{label}) RETURN count(n) AS count"
        )
        node_counts[label] = results[0]["count"] if results else 0
    
    return StatsResponse(
        labels=schema["labels"],
        relationship_types=schema["relationship_types"],
        node_counts=node_counts,
        index_count=len(schema["indexes"]),
        constraint_count=len(schema["constraints"])
    )


@app.get("/", tags=["Utility"])
async def root():
    """API root - returns welcome message."""
    return {
        "message": "Neo4j Knowledge Graph API",
        "version": "1.0.0",
        "docs": "/docs",
        "health": "/health"
    }


# ============================================================
# STARTUP/SHUTDOWN
# ============================================================

@app.on_event("startup")
async def startup_event():
    """Initialize database connection on startup."""
    get_db()
    print("✅ Database connection initialized")


@app.on_event("shutdown")
async def shutdown_event():
    """Close database connection on shutdown."""
    global _connection
    if _connection:
        _connection.close()
        _connection = None
    print("✅ Database connection closed")


# ============================================================
# RUN SERVER
# ============================================================

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)
```

## 30.2 API Usage Examples

```bash
# Health check
curl http://localhost:8000/health

# Create a patient
curl -X POST http://localhost:8000/nodes \
  -H "Content-Type: application/json" \
  -d '{"label": "Patient", "properties": {"id": "P100", "name": "API Patient"}}'

# Get a patient
curl http://localhost:8000/nodes/Patient/P100

# Update a patient
curl -X PUT http://localhost:8000/nodes/Patient/P100 \
  -H "Content-Type: application/json" \
  -d '{"properties": {"email": "api@example.com"}}'

# Create a relationship
curl -X POST http://localhost:8000/relationships \
  -H "Content-Type: application/json" \
  -d '{
    "from_label": "Patient",
    "from_id": "P100",
    "rel_type": "DIAGNOSED_WITH",
    "to_label": "Disease",
    "to_id": "E11",
    "properties": {"date": "2024-01-15"}
  }'

# Execute custom query
curl -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{
    "query": "MATCH (p:Patient)-[:DIAGNOSED_WITH]->(d:Disease) RETURN p.name, d.name LIMIT 5"
  }'

# RAG query
curl -X POST http://localhost:8000/rag \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What medications are used to treat Type 2 Diabetes?",
    "top_k": 5
  }'

# Get statistics
curl http://localhost:8000/stats

# Delete a patient
curl -X DELETE "http://localhost:8000/nodes/Patient/P100?detach=true"
```

---

# Appendix A: Complete Code Reference

## A.1 Project Structure

```
neo4j-knowledge-graph-tutorial/
├── README.md
├── COMPLETE_TUTORIAL.md          # This file
├── requirements.txt
├── docker-compose.yml
├── .env.example
│
├── src/
│   ├── utils/
│   │   ├── __init__.py
│   │   └── connection.py         # Database connection utilities
│   │
│   ├── 01_connection_basics.py   # Connection patterns
│   ├── 02_nodes_and_properties.py
│   ├── 03_relationships.py
│   ├── 04_basic_queries.py       # CRUD operations
│   ├── 05_advanced_queries.py    # Complex Cypher
│   ├── 06_indexing_and_constraints.py
│   ├── 07_data_loader.py         # Bulk data loading
│   ├── 08_transaction_management.py
│   ├── 09_batch_operations.py
│   ├── 10_gds_basics.py          # GDS fundamentals
│   ├── 11_centrality_algorithms.py
│   ├── 12_community_detection.py
│   ├── 13_similarity_algorithms.py
│   ├── 15_pathfinding_algorithms.py
│   ├── 18_ontology_design.py     # Knowledge graph ontology
│   ├── 19_populate_ontology.py
│   ├── 20_chunking_strategies.py # Document chunking
│   ├── 21_vector_embeddings.py
│   ├── 22_hnsw_indexing.py       # Vector search
│   ├── 23_graph_rag_basics.py    # Graph RAG
│   ├── 26_large_scale_clustering.py
│   └── 30_api_server.py          # REST API
│
├── data/
│   └── sample_healthcare_data.json
│
├── examples/
│   ├── healthcare_kg/
│   └── document_rag/
│
└── tests/
    └── test_connection.py
```

## A.2 Quick Start Commands

```bash
# 1. Clone or create project
mkdir neo4j-tutorial && cd neo4j-tutorial

# 2. Start Neo4j
docker-compose up -d

# 3. Setup Python environment
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate
pip install -r requirements.txt

# 4. Configure environment
cp .env.example .env
# Edit .env with your credentials

# 5. Test connection
python src/01_connection_basics.py

# 6. Load sample data
python src/07_data_loader.py

# 7. Run API server
uvicorn src.30_api_server:app --reload

# 8. Access:
# - Neo4j Browser: http://localhost:7474
# - API Docs: http://localhost:8000/docs
```

---

# Appendix B: Cypher Cheat Sheet

## B.1 CRUD Operations

```cypher
-- CREATE
CREATE (n:Label {prop: 'value'})
CREATE (a)-[:REL_TYPE {prop: 'value'}]->(b)

-- READ
MATCH (n:Label) RETURN n
MATCH (n:Label {id: 'value'}) RETURN n
MATCH (a)-[r:REL_TYPE]->(b) RETURN a, r, b

-- UPDATE
MATCH (n:Label {id: 'value'})
SET n.prop = 'new_value'

-- DELETE
MATCH (n:Label {id: 'value'}) DELETE n
MATCH (n:Label {id: 'value'}) DETACH DELETE n
```

## B.2 Pattern Matching

```cypher
-- Variable-length paths
(a)-[*1..3]->(b)      -- 1 to 3 hops
(a)-[*]->(b)          -- Any length
(a)-[:TYPE*2]->(b)    -- Exactly 2 hops of TYPE

-- Optional match
OPTIONAL MATCH (a)-[r]->(b)

-- Multiple patterns
MATCH (a:A), (b:B)
WHERE (a)-[:REL]->(b)
```

## B.3 Aggregations

```cypher
count(n)              -- Count
collect(n)            -- Create list
avg(n.prop)           -- Average
sum(n.prop)           -- Sum
min(n.prop), max(n.prop)
stDev(n.prop)         -- Standard deviation
percentileCont(n.prop, 0.5)  -- Median
```

## B.4 String Functions

```cypher
toLower(s), toUpper(s)
trim(s), ltrim(s), rtrim(s)
substring(s, start, length)
replace(s, old, new)
split(s, delimiter)
size(s)               -- String length
```

## B.5 List Functions

```cypher
size(list)
head(list), tail(list), last(list)
[x IN list WHERE condition | expression]
reduce(s = 0, x IN list | s + x)
```

## B.6 Indexes and Constraints

```cypher
-- Create range index
CREATE INDEX idx_name FOR (n:Label) ON (n.property)

-- Create fulltext index
CREATE FULLTEXT INDEX idx_name FOR (n:Label) ON EACH [n.prop1, n.prop2]

-- Create vector index
CREATE VECTOR INDEX idx_name FOR (n:Label) ON (n.embedding)
OPTIONS {indexConfig: {`vector.dimensions`: 384, `vector.similarity_function`: 'cosine'}}

-- Create unique constraint
CREATE CONSTRAINT constraint_name FOR (n:Label) REQUIRE n.property IS UNIQUE
```

---

# Appendix C: Troubleshooting Guide

## C.1 Common Issues

### Connection Refused
```
Error: ServiceUnavailable - Could not connect to localhost:7687

Solutions:
1. Check if Neo4j is running: docker-compose ps
2. Wait for startup: docker-compose logs -f neo4j
3. Verify port mapping in docker-compose.yml
4. Check firewall settings
```

### Authentication Failed
```
Error: AuthError - The client is unauthorized

Solutions:
1. Verify credentials in .env file
2. Reset password: docker exec -it neo4j-tutorial cypher-shell -u neo4j
3. Check NEO4J_AUTH in docker-compose.yml
```

### Out of Memory
```
Error: OutOfMemoryError

Solutions:
1. Increase heap size: NEO4J_dbms_memory_heap_max__size=4G
2. Increase page cache: NEO4J_dbms_memory_pagecache_size=4G
3. Use batch processing for large imports
4. Add more RAM to Docker
```

### GDS Not Available
```
Error: Unknown function 'gds.version'

Solutions:
1. Check plugin installation: docker exec neo4j-tutorial ls /plugins
2. Verify NEO4J_PLUGINS in docker-compose.yml
3. Restart container: docker-compose restart
4. Check procedure allowlist settings
```

### Slow Queries
```
Issue: Queries taking > 1 second

Solutions:
1. Profile query: PROFILE MATCH ... RETURN ...
2. Add indexes on filtered properties
3. Reduce result set with LIMIT
4. Use parameterized queries
5. Warm up page cache
```

## C.2 Performance Targets

| Operation | Target Time | Requirements |
|-----------|-------------|--------------|
| Indexed lookup | < 1 ms | Range index on property |
| 2-hop traversal | < 5 ms | Warm page cache |
| Vector search (10K) | < 10 ms | HNSW index |
| PageRank (10K) | < 500 ms | GDS graph projection |
| Bulk insert (1K) | < 100 ms | UNWIND batch pattern |

---

# Summary

This tutorial covered:

1. **Foundations (1-10)**: Connection, nodes, relationships, CRUD, queries, indexes
2. **Graph Data Science (11-17)**: Centrality, community, pathfinding, similarity, embeddings
3. **Knowledge Graphs (18-25)**: Ontology, chunking, vectors, HNSW, Graph RAG
4. **Production (26-30)**: Large-scale, internals, optimization, deployment, API

**Key Takeaways**:
- Neo4j's native graph storage enables O(1) traversals
- Indexes are critical for lookup performance
- GDS provides powerful graph analytics
- Vector indexes enable semantic search
- Graph RAG combines retrieval with graph context
- Proper configuration ensures 5ms query performance

**Next Steps**:
1. Practice with the sample healthcare data
2. Experiment with GDS algorithms
3. Build a Graph RAG application
4. Deploy with the FastAPI server
5. Scale using clustering strategies

---

*End of Tutorial*

*Total: ~100 pages / 5000+ lines*

